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SECTION  I 
INTRODUCTION 

The  Chief  of  Naval  Education  and  Training  (CNET)  requires  that  his  sub¬ 
ordinate  schools  provide  inservice  training  (1ST)  to  assigned  personnel. 

The  general  intent  of  1ST  is  to  enhance  and  maintain  personnel  skills  and 
knowledge  necessary  for  the  efficient  and  effective  management  and  delivery 
of  training.  Currently,  1ST  is  most  often  designed  at  the  local  level  to 
meet  needs  at  a  specific  activity.  Thus,  1ST  varies  in  content  and  emphasis 
across  Naval  Education  and  Training  Command  (NAVEDTRACOM)  activities.  Atten¬ 
tion  to  the  provision  of  1ST  on  ?.  command-wide  basis  has  typically  been 
directed  to  meeting  specific  needs  as  they  arise.  Systematic  attempts  to 
identify  and  contend  with  the,  perhaps,  more  enduring  and  critical  school 
staff  training  needs  have  been  lacking. 

CNET  tasked  the  Training  Analysis  and  Evaluation  Group  (TAEG)  to  assess 
1ST  needs  for  personnel  assigned  to  NAVEDTRACOM  training  activities*  with 
the  emphasis  on  command-wide  needs  rather  than  just  those  of  individual  train¬ 
ing  activities.  The  information  developed  by  TAEG  will  be  used  by  CNET  to 
plan  for  the  acquisition  of  training  materials  needed  to  support  1ST. 

PURPOSE  OF  THIS  STUDY 

The  purpose  of  this  study  was  to  (1)  assess  the  inservice  training  needs 
of  personnel  assigned  to  CNET  training  activities  and  (2)  develop  a  priori¬ 
tized  listing(s)  of  command-wide  inservice  needs  for  CNET  attention. 

Four  groups  of  training  activity  personnel  were  identified  by  TAEG  as 
potential  candidates  for  inservice  training: 

•  Curriculum  and  Instruction  Standards  Office  (CISO)  personnel 

•  Training  Executives  (i.e.,  commanding  officers,  executive  officers, 
and  directors  of  training) 

•  Training  Managers  (i.e.,  training  department  heads,  school/course 
heads  and  instructor  supervisors) 

•  Instructors. 

The  present  report  documents  inservice  requirements  for  the  CISO  personnel 
and  the  training  executive  groups.  A  companion  report  (Ford,  Whitten,  &  Hall, 
1983)  delineates  the  1ST  needs  of  training  managers  and  instructors. 


1CNET  ltr  Code  022  of  25  Sep  1981  (NOTAL). 
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SECTION  II 

THE  INSERVICE  CONCEPT 

This  section  addresses  the  concept  of  inservice  training.  Certain 
qeneral  considerations  relevant  to  the  concept  are  presented,  followed  by  a 
brief  description  of  current  inservice  requirements. 

GENERAL  CONSIDERATIONS 

1ST  is  formal  training  that  individuals  receive  after  reporting  for 
duty  at  a  training  activity.  The  training  is  formal  if  it  has  a  curriculum, 
objectives  to  be  achieved,  and  criteria  for  success  or  failure.  Inservice 
training  is  distinguished  from  on-the-job  training  in  that  1ST  occurs  during 
periods  when  individuals  are  not  involved  in  the  performance  of  their  normal 
job  duties. 

Most  inservice  training  assumes  that  assigned  personnel  are  basically 
qualified  for  the  positions  they  occupy.  Thus,  the  purpose  of  1ST,  generally, 
is  to  enhance  and/or  maintain  the  skills  of  currently  qualified  personnel 
rather  than  to  impart  basic  skills  to  previously  unqualified  personnel.  There 
are  at  least  three  applications  for  which  1ST  is  appropriate:  refresher 
training,  training  in  new  procedures  and  techniques,  and  training  in  special¬ 
ized  procedures  and  techniques.  For  these  purposes,  1ST  is  likely  to  be 
more  desirable  than  sending  personnel  to  a  centralized  school.-  A  centralized 
approach  would  require  more  time  and  money  and  would  also  remove  qualified 
personnel  from  their  duty  stations  during  training  periods.  One  of  the  advan¬ 
tages  inservice  training  is  that  personnel  being  trained  are  still  avail¬ 
able  to  the  activity  to  perform  their  duties.  Despite  its  advantages,  inser¬ 
vice  is  not  the  optimal  mode  for  all  kinds  of  training.  For  lengthy  programs 
in  particular,  centralized  training  may  have  advantages  such  as  requiring 
fewer  personnel  to  conduct  and  maintain  programs.  It  may  also  permit  more 
efficient  use  of  equipment  and  facilities. 

CURRENT  INSERVICE  REQUIREMENTS 

Official  publications  of  the  CNET  and  the  Chief  of  Naval  Technical  Train¬ 
ing  (CNTECHTRA)  refer  to  inservice  training. 

Procedures  for  Instructional  Systems  Development  (NAVEDTRA  110A)  provides 
guidance  ^or  the  analysis,  design,  development,  implementation,  and  control 
of  instructional  programs  under  CNET  cognizance.  The  need  for  inservice 
training  is  recognized  with  the  following  statement: 

INSERVICE  TRAINING.  All  activities  will  provide 
a  formal  inservice  training  program  monthly  for 
instructional  and  supervisory  personnel. 

CNET  Instruction  1540. 6A  (1982)  delegates  to  training  activity  CISOs 
the  responsibility  for  maintaining  the  quality  of  local  technical  training 
through  instructor  and  staff  inservice  training  programs. 


8 


Technical  Report  144 


CNTECHTRA  Instruction  1540.47  (1979)  provides  "policy  and  guidance  for 
the  conduct  of  an  instructor/staff  inservice  training  program"  for  training 
activities  under  CNTECHTRA  cognizance.  Several  guidelines  are  provided. 
Inservice  should  be  "formal,  scheduled,  and  periodically  evaluated."  The 
purpose  of  inservice  training  is  to  correct  deficiencies  and  provide  for 
professional  growth.  Inservice  programs  should  be  developed  with  the  guidance 
of  the  CISO.  Also,  not  only  are  instructors  to  be  given  inservice  training, 
but  so  should  staff  and  supervisory  personnel.  The  instruction  states  that 
needs  for  specific  inservice  training  can  be  determined  by  internal  evaluation 
and  day-to-day  supervision. 

The  documents  cited  above  establish  a  formal  requirement  for  inservice 
to  be  conducted  at  training  activities.  The  content  of  inservice  is,  however, 
larqely  left  to  the  discretion  of  each  local  activity.  Currently,  there  is 
a  lack  of  information  concerning  inservice  needs  across  the  Naval  Education 
and  Training  Command  (NAVEDTRACOM) .  The  present  study  was  designed  to  assess 
NAVEDTRACOM-wide  needs.  The  technical  approach  taken  to  assess  these  needs 
is  described  in  the  next  section  of  this  report. 
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SECTION  III 
TECHNICAL  APPROACH 

This  section  presents  the  technical  approach  used  to  assess  inservice 
training  needs  of  CISO  personnel  and  training  executives  (i.e.,  commanding 
officers,  executive  officers,  and  directors  of  training)  across  the 
NAVEDTRACOM.  The  procedural  steps  consisted  of  project  planning  and 
coordination,  questionnaire  development,  designing  study  samples,  data 
collection,  and  data  analysis.  Each  of  these  steps  is  discussed  below. 

PROJECT  PLANNING  AND  COORDINATION 

At  the  beginning  of  the  project,  planning  and  coordination  meetings 
were  held  with  selected  training  activity  personnel  and  with  CNET  and 
CNTECHTRA  staff. 


Open-ended  interviews  were  conducted  with  training  activity  personnel 
to  obtain  information  concerning: 

•  the  functions  performed  at  the  activity  by  various  personnel  groups 

•  the  existence  and  success  of  formal  inservice  training  programs  at 
that  activity 

•  the  target  populations  of  those  programs 

•  who  designs,  delivers,  and  evaluates  inservice  programs 

•  the  inservice  needs  of  assigned  staff. 

Information  obtained  from  these  interviews  was  used  in  project  planning  to 
delimit  the  inservice  training  study  and  as  input  to  data  collection 
instruments. 


During  individual  meetings  with  CNET  and  CNTECHTRA  staff  personnel, 
various  inputs  to  the  program  were  received.  At  CNTECHTRA,  a  general  staff 
briefing  was  provided  by  the  TAEG  project  team.  Subsequently,  open-ended 
interviews  were  conducted  with  groups  of  Training  Program  Coordinators 
(TPCs).  At  these  sessions,  TPCs  provided  inputs  concerning  the  adequacy  of 
function  lists  generated  through  interviews  with  training  activity  personnel 
and/or  extracted  from  other  sources.  The  TPCs  also  assisted  the  TAEG 
project  staff  in  selecting  a  representative  sample  of  training  activities 
from  which  data  would  be  collected. 


To  ensure  maximum  participation  in  the  study  by  personnel  at  the  train¬ 
ing  activities  of  interest,  CNET  transmitted  letters  to  the  CNET  functional 
commanders  and  to  the  Naval  Education  and  Training  Center  requesting  support 
for  the  study. 2  The  functional  commanders,  in  turn,  requested  maximum  par¬ 
ticipation  from  their  subordinate  activities. 


2 CNET  ltr  Code  00A2  of  8  Feb  1982. 
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QUESTIONNAIRE  DEVELOPMENT 

To  minimize  project  costs,  questionnaires  were  used  to  obtain  informa¬ 
tion  from  CISO  personnel  and  training  executives. 

The  first  step  in  questionnaire  development  was  to  determine  the  func¬ 
tions  that  were  performed  by  each  particular  type  of  personnel.  The  findinqs 
of  a  recent  study  of  CISO  personnel  and  their  functions  (Hall,  Ford,  & 
Middleton,  1981)  were  used  to  create  an  initial  list  of  CISO  functions. 

Various  general  sources  were  used  to  identify  functions  performed  by  train¬ 
ing  executives.  Information  extracted  from  these  general  sources  was  supple¬ 
mented  by  interviews  conducted  with  incumbent  training  executives.  In  addition 
to  identifying  functions  that  training  executives  perform,  areas  in  which 
specialized  knowledge  might  be  required  were  also  identified. 

The  lists  of  functions  and  knowledge  areas  were  subsequently  reviewed 
by  personnel  knowledgeable  of  training  activity  operations  (e.g.,  TPCs  and 
education  specialists)  to  determine  if  there  were  any  significant  omissions 
or  inappropriate  inclusions. 

The  resulting  CISO  questionnaire  had  two  sections;  the  training  execu¬ 
tive  questionnaire  had  three.  A  copy  of  each  questionnaire  is  shown  in 
appendix  A.  In  both  cases,  the  first  section  dealt  with  general  respondent 
information  and  also  asked  questions  about  the  degree  to  which  the  incumbent's 
job  had  changed  during  his/her  tenure.  Again,  in  both  cases,  the  second 
section  was  concerned  with  specific  job  functions.  For  each  function,  six 
questions  were  asked: 

•  On  the  average,  how  often  do  you  perform  this  function? 

•  How  difficult  is  it  for  you  to  perform  this  function? 

•  When  you  first  arrived  at  this  activity,  how  difficult  was  it  for 
you  to  perform  this  function? 

•  How  important  is  the  performance  of  this  function  to  success  at 
your  job? 

•  Did  you  receive  any  formal  inservice  training  in  how  to  perform 
this  function  when  you  arrived  at  this  activity? 

•  If  not,  how  useful  would  inservice  training  in  how  to  perform  this 
function  have  been  when  you  first  arrived  at  this  activity? 

Those  interested  in  the  specific  response  options  for  each  question  should 
refer  to  appendix  A. 

The  questions  listed  above  were  used  because  they  form  the  foundation 
of  a  training  task  analysis.  Similar  questions  have  been  used  previously 
(Hall,  Ford,  and  Middleton,  1981;  Hughes,  Ford,  Heidt,  and  Copeland,  1981) 
to  analyze  specific  jobs  and  to  assess  training  needs  of  job  incumbents. 
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The  third  section  of  the  training  executive  questionnaire  was  concerned 
with  knowledge  areas.  Five  questions  were  asked  about  each  knowledge  area 
listed: 

•  How  familiar  are  you  with  this  area? 

•  How  important  is  familiarity  with  this  area  to  success  at  your 
job? 

•  How  familiar  were  you  with  this  area  when  you  first  arrived  at 
this  activity? 

•  Did  you  receive  any  formal  inservice  training  in  this  area  when 
you  first  arrived  at  this  activity? 

•  If  not,  how  useful  would  inservice  training  in  this  area  have  been 
when  you  first  arrived  at  this  activity? 

To  establish  training  priorities  for  tasks  that  constitute  a  job,  three 
basic  items  of  information  are  required: 

•  Frequency  of  performance  of  a  function 

•  Importance  of  performance  of  a  function 

•  Difficulty  of  performance  of  a  function. 

The  frequency  of  performance  of  a  function  is  self-explanatory.  It 
simply  means  how  often  an  individual  does  a  particular  thing.  Difficulty  of 
performance  is  also  self-explanatory.  The  term  "importance"  is  essentially 
synonymous  to  tne  term  "criticality,"  which  is  more  frequently  used  in  opera¬ 
tional  contexts.  Criticality  indicates  the  cost  of  failure  to  perform  a 
function  correctly.  Aircraft  maintenance  functions,  for  example,  may  have 
an  extremely  high  criticality.  Thus,  even  if  a  particular  function  is  per¬ 
formed  infrequently,  it  may  be  extremely  important  that  it  be  done  correctly 
when  it  is  done. 

The  priority  of  training  for  particular  functions  should  be  established 
on  the  basis  of  a  combination  of  these  three  dimensions.  Those  functions 
that  are  frequently  performed,  very  important,  and  very  difficult  would  be 
given  the  highest  priority  for  training.  Likewise,  those  functions  that  are 
infrequently  performed,  are  not  important,  and  are  easily  done  would  be  given 
the  lowest  priority  for  training.  Training  priorities  for  functions  that 
fall  in  the  middle  could  be  ordered  using  a  variety  of  rules  for  combining 
the  three  dimensions. 

Questions  concerning  difficulty  of  performance  of  particular  functions 
upon  arrival  at  a  training  activity  were  included  on  the  questionnaires  to 
determine  if  there  were  differences  in  inservice  needs  for  personnel  when 
they  first  arrived  compared  to  the  current  time.  Also,  since  this  study  was 
oriented  to  a  particular  type  of  training  (i.e.,  inservice)  respondents  were 
specifically  asked  about  the  usefulness  of  inservice.  Finally,  to  assess 
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the  extent  of  current,  formal  inservice  training  within  the  NAVEDTRACOM, 
respondents  were  asked  if  they  had  received  any  formal  inservice  in  each 
function  upon  arrival  at  their  duty  station. 

SAMPLING  STRATEGY 

Sampling  for  this  study  required  two  steps.  First,  the  population  of 
training  activities  had  to  be  determined  and  sampled.  Second,  personnel 
within  each  training  activity  had  to  be  sampled. 

To  determine  the  population  of  training  activities,  a  list  of  all  staff 
unit  identification  codes  (UIC)  was  compiled  for  all  Navy  courses  listed  in 
the  Navy  Integrated  Training  Resources  and  Administrative  System  (NITRAS). 
Next,  this  list  of  staff  UICs  was  compared  with  the  Navy  Comptroller's  Manual, 
Vol.  2,  Chapter  5,  Revision  42,  to  identify  all  Navy  activities  that  were 
conducting  training.  There  were  over  100  activities  identified.  The  next 
step  in  reducing  the  population  was  to  limit  it  to  CNET  activities.  The 
reduced  list  was  then  presented  to  TPCs  at  CNTECHTRA  with  the  request  that 
they  nominate  as  few  activities  as  possible  that  would  be  representative  of 
all  CNET  training  activities  on  the  list.  The  resulting  sample  of  35  train¬ 
ing  activities  and  detachments  is  shown  in  table  1.  Although  the  sample  of 
activities  was  not  randomly  drawn,  there  is  no  reason  to  believe  that  it  was 
not  representative.  Further,  TPCs  who  had  intimate  knowledge  of  the  activi¬ 
ties  involved  agreed  that  the  sample  did  adequately  represent  the  larger 
group. 

With  the  sample  of  activities  drawn,  it  was  then  necessary  to  determine 
how  to  sample  individuals  within  activities.  As  a  preliminary  step  in  this 
process,  a  message3  was  sent  to  all  sampled  training  activities.  Each 
activity  was  requested  to  provide  the  numbers  of  personnel  at  the  activity 
who  occupied  specific  billets/positions.  Subsequently,  a  sampling  strategy 
was  selected  and  a  procedure  for  distributing  questionnaires  was  enclosed 
with  each  packet  of  questionnaires  sent  to  an  activity  (see  appendix  A). 

For  the  CISO  group  and  the  training  executive  group,  all  assigned  personnel 
were  sampled. 

DATA  COLLECTION 

Questionnaire  packets  were  mailed  to  the  training  activities  on  8  April 
1982.  To  promote  candor,  respondents  were  requested  to  return  the  completed 
questionnaires  directly  to  the  TAEG  rather  than  through  command  channels. 
Preaddressed  envelopes  were  provided  for  this  purpose.  It  was  further 
requested  that  each  training  activity  return  all  unneeded,  excess  question¬ 
naires,  so  marked,  to  the  TAEG. 


3TAEG  msg  091450Z  MAR  82. 
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TABLE  1.  TRAINING  ACTIVITIES  AND  DETACHMENTS  SAMPLED 


Naval  Diving  and  Salvage  Training  Center,  Panama  City 

Naval  Air  Maintenance  Training  Group  Detachment,  North  Island 

Naval  Amphibious  School,  Coronado 

Naval  School,  Explosive  Ordnance  Disposal,  Indian  Head 

Naval  Air  Technical  Training  Center,  Millington 

Service  School  Command,  Orlando 

Naval  Submarine  Training  Center,  Pacific 

Service  School  Command,  San  Diego 

Submarine  Training  Facility,  San  Diego 

Naval  Technical  Training  Center,  Corry  Station 

Naval  Damage  Control  Training  Center,  Philadelphia 

Naval  Air  Technical  Training  Center,  Lakehurst 

Naval  Technical  Training  Center,  Meridian 

Naval  Supply  Corps  School,  Athens 

Naval  Justice  School,  Newport 

Naval  Air  Maintenance  Training  Group  Detachment,  Oceana 

Naval  Construction  Training  Center,  Port  Hueneme 

Service  School  Command,  Great  Lakes 

Naval  Submarine  School,  Groton 

Naval  Air  Maintenance  Training  Group,  Millington 

Trident  Training  Facility,  Bangor 

Combat  Systems  Technical  School  Command,  Mare  Island 

Naval  Technical  Training  Center,  Treasure  Island 

Naval  School,  Civil  Engineer  Corps  Officers,  Port  Hueneme 

Naval  Construction  Training  Center,  Gulfport 

Surface  Warfare  Officers  School  Command,  Newport 

Human  Resource  Management  School,  Millington 

Naval  Education  and  Training  Center,  Newport 

Fleet  Anti-Submarine  Warfare  Training  Center,  Atlantic 

Fleet  Training  Center,  Norfolk 

Fleet  Combat  Training  Center,  Atlantic 

Fleet  and  Mine  Warfare  Training  Center,  Charleston 

Fleet  Anti-Submarine  Warfare  Training  Center,  Pacific 

Fleet  Training  Center,  San  Diego 

Fleet  Combat  Training  Center,  Pacific 


DATA  PROCESSING  AND  ANALYSIS 

As  the  questionnaires  were  returned  by  mail,  they  were  entered  into 
computer  storage.  The  Statistical  Package  for  the  Social  Sciences  (Nie, 
Hull,  Jenkins,  Steinbrenner,  and  Bent,  1975)  software  package  was  used  for 
data  management  and  analysis.  The  computational  facilities  of  the  Northeast 
Regional  Data  Center  of  the  State  University  System  of  Florida,  located  at 
the  University  of  Florida,  Gainesville,  Florida,  and  of  the  University  of 
Central  Florida,  Orlando,  Florida,  were  used. 
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SECTION  IV 
RESULTS 

This  section  presents  the  results  of  the  analysis  of  questionnaire 
responses  to  the  CISO  and  Training  Executive  Inservice  Training  Needs 
Assessment  Questionnaires. 

RETURN  RATES 

For  the  entire  study  of  all  four  groups  of  personnel,  1,623  question¬ 
naires  were  distributed  and  1,343  usable  questionnaires  were  returned,  for 
an  overall  response  rate  of  83  percent. 

A  total  of  244  CISO  questionnaires  were  distributed  to  CISO  personnel 
and  199  were  returned,  for  a  response  rate  of  82  percent.  Of  the  199  usable 
CISO  returns,  57  were  from  civilians,  23  were  from  commissioned  officers, 
and  116  were  from  enlisted  personnel.  Three  respondents  did  not  identify 
themselves  or  fell  into  other  categories  (e.g.,  warrant  officers).  Of  the 
57  civilians,  34  identified  themselves  as  education  specialists  and  14  iden¬ 
tified  themselves  as  training  specialists. 

A  total  of  87  training  executive  questionnaires  were  distributed  and  76 
were  returned,  for  a  response  rate  of  87  percent.  Of  the  76  returns,  29 
were  from  commanding  officers  (CO),  22  were  from  executive  officers  (XO),  20 
were  from  directors  of  training  (DOT).  Five  did  not  place  themselves  in  a 
category. 

ANALYTIC  STRATEGY 

In  order  to  determine  the  priority  of  need  for  inservice  training, 
several  preliminary  analytic  steps  were  performed.  First,  the  responses  to 
frequency  of  performance  of  individual  function  items  were  factor  analyzed 
for  both  training  executives  and  CISO  personnel.  Based  on  these  factor 
analyses,  major  function  groups  were  identified.  Second,  scales  consisting 
of  items  comprising  major  function  groups  were  constructed  for  frequency, 
difficulty,  and  importance  of  performance,  and  usefulness  of  inservice. 

These  major  function  groups  served  as  the  basis  for  further  analysis, 
including  the  current  extent  of  1ST,  differences  among  groups  of  personnel 
in  types  of  major  functions  performed,  and,  finally,  in  the  development  of 
priorities  for  1ST.  Priorities  were  developed  separately  for  groups  of 
personnel  identified  as  having  different  functional  requirements. 

MAJOR  FUNCTION  GROUPS 

The  first  step  in  the  analysis  was  to  determine  if  specific  function 
items  could  be  collected  into  major  function  groups.  Two  steps  were  required 
to  establish  these  groups.  First,  specific  function  items  were  grouped  by 
similarities  in  frequency  of  performance.  Frequencies  were  used,  instead  of 
difficulty  or  importance  of  performance,  since  frequency  would  reflect  a 
type  of  job.  Second,  the  same  function  items  grouped  by  frequency  were  used 
to  build  scales  for  the  other  questions  (e.g.,  difficulty  and  importance  of 
performance).  This  strategy  was  followed  in  analyzing  both  the  CISO  and  the 
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training  executive  data.  The  results  of  the  factor  analyses  used  to  determine 
major  function  groups  are  reported  in  appendix  B. 

FACTOR  ANALYSIS  OF  CISO  FUNCTIONS.  From  the  factor  analysis  of  CISO  personnel, 
four  major  function  groups  were  identified.  Names  or  labels  were  assigned 
to  each  group  based  on  inspection  of  the  specific  function  items  constituting 
a  major  function  group. 

The  sets  of  items  constituting  each  major  function  group  were  again 
factor  analyzed  to  determine  if  there  were  meaningful  subgroups.  For  the 
factor  solutions,  see  appendix  B.  The  four  major  function  groups,  with  two 
subgroups  for  each,  are  listed  in  table  2.  The  four  major  function  groups 
for  CISOs  are: 


•  Preliminary  course  development 

•  Secondary  course  development 

•  Evaluation 

•  Service  to  Activity 

Preliminary  course  development  is  comprised  of  10  functions  falling 
into  two  subgroups  based  on  the  second  factor  analysis.  Overall,  prelim¬ 
inary  course  development  corresponds  roughly  to  the  first  two  steps  of  the 
Instructional  Systems  Development  (ISO)  model  (NAVEDTRA  110A).  The  first 
subgroup  represents  the  Analysis,  the  second  the  Design  phase  of  ISO. 

Secondary  course  development  is  composed  of  five  functions.  The  first 
four  functions,  listed  as  subgroup  1,  correspond  to  the  development  phase  of 
ISD.  The  fifth  function,  assisting  with  student  testing,  is  somewhat  of  a 
surprise  since  it  was  expected  to  fall  under  the  evaluation  factor. 

The  major  function  of  evaluation  comprises  seven  specific  functions 
(table  2).  This  major  function,  corresponding  to  the  control  phase  of  ISD, 
can  be  broken  down  into  two  subgroups,  representing  internal  and  external 
evaluation. 

The  final  major  function  for  CISO  personnel  is  called  service  to  activity. 
The  six  specific  functions,  shown  in  table  2,  can  be  grouped  into  inservice 
and  coordination  functions.  This  major  function  does  not  correspond  to  any 
particular  phase  of  ISD.  It  could  be  considered  as  a  general  training  support 
function  since  its  basic  thrust  is  to  assist  training  activity  personnel  to 
accomplish  the  training  mission  more  readily. 

The  subgroups  of  the  four  major  function  areas,  all  based  on  secondary 
factor  analyses,  are  presented  primarily  as  an  aid  to  curriculum  developers 
who  might  use  this  list  of  functions  to  develop  1ST  programs.  While  the 
major  function  areas  constitute  topics  of  an  inservice  program,  the  subgroups 
constitute  subtopics,  and  the  specific  function  items  indicate  the  appropriate 
content  for  the  topics  and  subtopics. 
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PRELIMINARY  COURSE  DEVELOPMENT 


Analyze  job/new  requirements 
Select  tasks  to  be  trained 
Select  job  performance  measures 
Analyze  existing  courses/modules 
Select  instructional  setting 

Develop  learning  objectives 
Develop  test  items  to  measure  objectives 
Describe  entry  level  behavior 
Establish  sequence  of  objectives 
Specify  learning  events/activities 

SECONDARY  COURSE  DEVELOPMENT 


Specify  instruction  management  plan  and  media 
Select  existing  materials 
Develop  instruction 
Validate  instruction 

Assist  with  student  testing 

EVALUATION 


Study  attrition  and  setbacks 

Design  instruments  for  internal  evaluation 

Collect  data  for  internal  evaluation 

Analyze  data  from  internal  evaluation 

Make  recommendations  based  on  internal  evaluation 

Assist  with  preparation  of  items  for  external  evaluation 
Assist  with  interpretation  of  results  of  external  evaluation 

SERVICE  TO  ACTIVITY 


Design  interdepartmental  inservice  programs 
Conduct  interdepartmental  inservice  programs 
Assist  with  intradepartmental  inservice  programs 

Coordinate  with  external  activities  and  agencies  concerned  with 
training  quality  assurance 

Coordinate  accreditation  requirements  and  review 
Advise  commanding  officer  on  CISO  civilian  affairs 
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FACTOR  ANALYSIS  OF  TRAINING  EXECUTIVE  FUNCTIONS.  From  the  factor  analysis 
of  training  executives,  four  major  function  groups  were  identified.  Again, 
names  or  labels  were  assigned  to  each  group  of  items.  The  components  of  the 
major  training  executive  function  groups  are  shown  in  table  3.  The  four 
major  function  groups  are: 

•  Quality  Assurance 

•  Personnel  Assessment 

•  Management  of  Student  Throughput 

•  Personal  Counseling. 


TABLE  3.  COMPONENTS  OF  THE  MAJOR  FUNCTION  GROUPS  FOR  TRAINING  EXECUTIVES 


QUALITY  ASSURANCE 

Implement  ‘.nternal  system  of  accountability  for 
training  quality 

Select  criteria  to  monitor  for  training  quality 

Determine  corrective  action  if  a  criterion  falls 
below  desirable  level 

Allocate  funds  internally  for  efficient  achieve¬ 
ment  of  training  quality 

Plan  for  future  support  needs 


PERSONNEL  ASSESSMENT 

Assess  subject  matter  expertise  of  personnel 
Assess  instructional  ability  of  personnel 
Assess  personnel  ability  to  design  curricula 


MANAGEMENT  OF  STUDENT  THROUGHPUT 
Manage  student  pipeline 

Minimize  delays  in  training  for  students  arriving 
at  activity 

Counsel  students  on  academic  matters 


PERSONAL  COUNSELING 

Recognize  drug/alcohol  abuse  and  take  appropriate 
action 

Counsel  staff  and  students  on  personal  matters 
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Quality  assurance  consists  of  five  functions  concerned  with  determining/ 
ensuring  current  and  future  quality  of  training. 

Personnel  assessment,  comprising  three  specific  functions,  concerns  the 
evaluation  of  the  capabilities  of  instructional  personnel. 

Managing  student  throughput  deals  with  student  scheduling  and  academic 
counseling. 

The  major  function  of  personal  counseling  concerns  assisting  students 
and  staff  with  personal  problems. 

The  major  functions  for  training  executives  were  not  factor  analyzed 
again.  The  number  of  items  in  each  was  too  small  to  allow  for  meaningful 
subgroups. 

SCALE  DEVELOPMENT 

To  this  point,  major  function  groups  have  been  established  for  CISO 
personnel  and  training  executives  based  on  frequency  of  performance  of 
specific  function  items.  The  next  step  is  to  determine  the  extent  to  which 
attributes  of  these  major  function  groups  (i.e.,  the  frequency,  difficulty 
and  importance  of  performance,  and  the  usefulness  of  in-service  training) 
can  be  measured  reliably  using  scales  created  from  individual  items  on  the 
questionnaires. 

A  scale  is  a  measure  made  of  several  specific  items.  It  is  usually  a 
better  measure  than  any  one  of  the  single  items  since  single  items  usually 
do  not  adequately  measure  complex  variables  (Babbie,  1973).  An  overall  test 
score  is  essentially  a  scale  constructed  of  scores  on  single  items  on  the 
test.  Likewise,  scales  can  be  constructed  of  single  questionnaire  items. 
However,  to  use  a  scale  made  of  several  items  as  a  variable  to  analyze,  its 
internal  consistency  reliability  should  be  established.  This  subsection 
describes  the  development  of  scales  and  establishment  of  their  internal  con¬ 
sistency  reliability. 

The  results  of  the  factor  analyses  of  frequency  of  performance  of  func¬ 
tion  items  were  used  to  develop  the  major  function  areas  for  CISO  personnel 
and  for  traininq  executives  separately.  The  same  specific  function  items 
that  constituted  major  function  areas  for  frequency  were  used  to  create  scales 
for  the  other  questions  on  the  questionnaires  (e.g.,  difficulty,  importance, 
usefulness  of  inservice).  Each  scale  developed  consists  of  the  mean  of  the 
responses  to  all  the  specific  function  items  that  constituted  the  major  func¬ 
tion  area  for  a  particular  question  (e.g.,  how  important  or  difficult).  Thus, 
for  example,  each  CISO  respondent  received  as  a  score  on  the  frequency  of 
preliminary  course  development  his  or  her  mean  score  on  the  10  frequency  of 
performance  items  listed  under  preliminary  course  development  in  table  2. 

For  each  of  the  scales  developed,  the  internal  consistency  coefficient 
alpha  (Cronbach,  1951)  was  computed.  This  coefficient  indicates  the  internal 
reliability  of  the  scale,  or  the  degree  to  which  each  of  the  items  constitut¬ 
ing  a  scale  measures  the  same  underlying  dimension.  The  internal  consistency 
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reliability  coefficients  are  shown  in  table  4  for  the  20  major  function  scales 
developed  for  CISO  respondents.  The  reliability  coefficients  for  the  20 
major  function  scales  for  training  executives  are  shown  in  table  5.  As  the 
coefficients,  with  the  exception  of  one,  are  all  above  .60,  the  reliabilities 
are  satisfactory.  Thus,  these  scales  are  reliable  measures  of  the  frequency, 
difficulty  and  importance  of  performance,  and  the  usefulness  of  inservice 
training  in  the  four  major  function  groups  of  CISO  personnel  and  in  the  four 
major  function  groups  of  training  executives. 

OTHER  FACTORS  IN  INSERVICE  TRAINING 

Section  I  of  both  the  CISO  and  training  executive  questionnaires  requested 
certain  background  information  from  respondents  and  also  asked  about  the 
extent  to  which  facets  of  each  respondent's  job  had  changed  since  he  or  she 
beqan  working  in  that  position.  Also,  COs  were  asked  if  they  had  attended 
the  Prospective  Commanding  Officer  (PCO)  Shore  Station  Management  Course 
offered  by  the  Personnel  Management  Institute,  Naval  Civilian  Personnel  Command. 
In  section  III  of  the  training  executive  questionnaire,  training  executives 
were  asked  about  areas  of  knowledge.  Results  obtained  from  these  items  are 
summarized  below. 

BACKGROUND  VARIABLES.  In  general,  the  relationships  between  background  vari¬ 
ables  and  major  function  attributes  were  not  significant  for  either  CISO 
personnel  or  training  executives,  when  corrections  for  simultaneous  estima¬ 
tion  were  used  (Wonnacott  and  Wonnacott,  1972).  Specifically,  level  of 
education,  previous  job  experience,  time  in  position,  and  changes  in  the 
nature  of  job  requirements  were  not  significantly  related  to  frequency, 
difficulty,  or  importance  of  performance  of  major  functions  of  CISO  per¬ 
sonnel  or  training  executives.  Neither  were  they  related  to  usefulness  of 
inservice  in  these  major  functions. 

PROSPECTIVE  COMMANDING  OFFICER  SHORE  STATION  MANAGEMENT  COURSE.  Training 
executives  were  asked  if  they  had  attended  the  PCO  Shore  Station  Management 
Course.  This  course  addresses  many  topics  relevant  to  the  jobs  of  NAVEDTRACOM 
training  executives.  Of  13  training  executives  who  attended,  12  recommended 
it. 

TRAINING  EXECUTIVE  AREAS  OF  KNOWLEDGE.  In  addition  to  questions  about  perfor¬ 
mance  of  functions,  training  executives  were  also  asked  about  their  familiarity 
with  12  areas  of  knowledge.  Specifically,  they  were  asked  how  familiar  they 
were  with  each  area  when  they  arrived  at  their  current  duty  station,  how 
important  familiarity  is,  whether  they  received  inservice  in  each  area,  and 
how  useful  inservice  would  have  been. 

There  was  no  meaningful  grouping  of  areas  of  knowledge.  Several  factor 
analyses  were  performed;  none  disclosed  meaningful  structures.  Therefore, 
the  individual  area  of  knowledge  items  for  training  executives  were  rank 
ordered  separately  for  commanding  officers,  executive  officers,  and  directors 
of  training.  The  rank  orders  were  then  used  to  indicate  inservice  training 
priorities.  Three  questions,  concerning  the  degree  of  familiarity  with  the 
areas  of  knowledge,  the  importance  of  familiarity,  and  the  usefulness  of  1ST 
in  the  areas,  were  used  to  establish  the  rank  orders. 
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TA8LE  4.  INTERNAL  RELIABILITY  COEFFICIENTS  OF  THE  MAJOR 
FUNCTION  SCALES  FOR  CISO  PERSONNEL 


ALPHA 

Frequency  of  Performance 


Preliminary  Development  .89 
Secondary  Development  .71 
Evaluation  .87 
Service  to  Activity  .72 

Difficulty  of  Performance 

Preliminary  Development  .95 
Secondary  Development  .87 
Evaluation  .91 
Service  to  Activity  .94 

Difficulty  of  Performance  Upon  Arrival 

Preliminary  Development  .94 
Secondary  Development  .93 
Evaluation  .95 
Service  to  Activity  .94 

Importance  of  Performance 

Preliminary  Development  .89 
Secondary  Development  .83 
Evaluation  .90 
Service  to  Activity  .87 

Usefulness  of  Inservice 

Preliminary  Development  .97 
Secondary  Development  .93 
Evaluation  .96 
Service  to  Activity  .93 


Technical  Report  144 


TABLE  5.  INTERNAL  RELIABILITY  COEFFICIENTS  OF  THE  MAJOR 
FUNCTION  SCALES  FOR  TRAINING  EXECUTIVES 


ALPHA 

Frequency  of  Performance 


Quality  Assurance  .83 
Personnel  Assessment  .83 
Management  of  Throughput  .79 
Personal  Problems  .52 

Difficulty  of  Performance 

Quality  Assurance  .81 
Personnel  Assessment  .78 
Management  of  Throughput  .69 
Personal  Problems  .63 

Difficulty  of  Performance  Upon  Arrival 

Quality  Assurance  .90 
Personnel  Assessment  .70 
Management  of  Throughput  .81 
Personal  Problems  .64 

Importance  of  Performance 

Quality  Assurance  .76 
Personnel  Assessment  .84 
Manaqement  of  Throughput  .76 
Personal  Problems  .74 

Usefulness  of  Inservice 

Quality  Assurance  .88 
Personnel  Assessment  .91 
Management  of  Throughput  .74 
Personal  Problems  .77 
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The  first  step  in  rank  ordering  the  areas  of  knowledge  was  to  reduce 
them  to  a  manageable  number.  For  each  of  the  three  questions  used,  the  areas 
of  knowledge  were  broken  down  into  three  categories  on  the  basis  of  low, 
medium,  and  high  scores.  Items  in  the  highest  third  were  assigned  a  score 
of  "1,"  items  in  the  middle  third  were  assigned  a  score  of  “2,"  and  items  in 
the  bottom  third  were  assigned  a  score  of  "3." 

The  second  step  in  rank  ordering  areas  of  knowledge  was  to  add  up  the 
scores  for  each  area  on  each  of  the  three  questions.  This  sum  indicated  the 
overall  rank  of  each  area  on  the  three  questions.  For  each  group  of  personnel 
(commanding  officers,  executive  officers,  and  directors  of  training)  the 
rank  ordered  areas  of  knowledge  are  shown  in  tables  6,  7,  and  8.  In  each 
table,  the  areas  of  knowledge  are  grouped  when  the  overall  rank  is  tied  (i.e., 
all  items  in  the  first  group  received  the  same  score  and  so  on). 


TABLE  6.  PRIORITIES  FOR  INSERVICE  TRAINING  OF  AREAS  OF 
KNOWLEDGE  FOR  COMMANDING  OFFICERS 


First 


Reporting  systems,  including  NITRAS,  MILPERSIS,  SHOROCS 
and  SHORSTAMPS 

Manpower  planning  for  training  activities 

PPBS  -  the  Planning,  Programming  and  Budgeting  System 

How  curriculum  is  approved 

CNET  training  equipment  support 


Second 


Principles  and  practices  of  ISO 
Organization  of  training  command 

Civil  Service  rules  and  regulations,  including  the  merit 
pay  system 

The  UCMJ  at  shore  activities 


Third 


Subject  matter  taught  at  activity 
Administrative  discharge  procedures 


Fourth 


Special  terminology  used  in  training 
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TABLE  7.  PRIORITIES  FOR  IN-SERVICE  TRAINING  OF  AREAS  OF 
KNOWLEDGE  FOR  EXECUTIVE  OFFICERS 


First 

Reporting  systems 
Second 

Civil  Service  rules  and  regulations 
Manpower  planning 

Third 

PPBS  -  the  Programming,  Planning  and  Budgeting  System 
Fourth 

Principles  and  practices  of  ISD 
The  UCMJ  at  shore  activities 
How  curriculum  is  approved 
CNET  training  equipment  support 

Fifth 

Subject  matter  taught  at  activity 
Organization  of  the  training  command 

Sixth 

Special  terminology  used  in  training 
Administrative  discharge  procedures 
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TABLE  8.  PRIORITIES  FOR  IN-SERVICE  TRAINING  OF  AREAS  OF 
KNOWLEDGE  FOR  DIRECTORS  OF  TRAINING 


First 

Principles  and  practices  of  ISD 
Organization  of  the  training  command 
Manpower  planning 
How  curriculum  is  approved 
CNET  training  equipment  support 

Second 

Subject  matter  taught  at  activity 
Civil  Service  rules  and  regulations 
Reporting  systems 

Third 

Special  terminology  used  in  training 

PPBS  -  the  Programming,  Planning  and  Budgeting  System 

Fourth 

The  UCMJ  at  shore  activities 
Administrative  discharge  procedures 


EXTENT  OF  INSERVICE  TRAINING 

To  estimate  the  degree  to  which  inservice  training  is  offered  currently 
within  the  NAVEDTRACOM,  CISO  personnel  and  training  executives  were  asked  if 
they  received  inservice  for  specific  function  items  when  they  arrived  at  the 
activity. 

CISO  PERSONNEL.  The  percentages  of  CISO  personnel  who  reported  they  had 
received  1ST  are  shown  in  table  9.  An  average  of  21  percent  of  the  CISO 
respondents  overall  reported  inservice  in  particular  functions.  On  the 
average,  officers  reported  inservice  in  12  percent  of  the  specific  func¬ 
tions,  enlisted  personnel  and  civilians  reported  inservice  in  22  percent. 
Preliminary  development  was  the  area  in  which  most  1ST  was  received  by  all 
three  groups,  followed  in  order  by  secondary  development,  evaluation,  and 
service  to  the  activity.  Generally,  however,  it  should  be  noted  that,  while 
1ST  is  conducted,  it  is  not  widespread.  Fewer  than  a  third  of  the  CISO  per¬ 
sonnel  in  any  group  reported  inservice  in  any  of  the  four  major  function 
areas. 


The  average  percent  of  specific  CISO  function  items  covered  by  1ST  varies 
from  activity  to  activity.  For  preliminary  development  functions,  only  one 
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activity  provides  inservice  in  all  the  specific  function  items.  At  five 
activities,  no  inservice  is  provided  in  any  of  the  items.  For  secondary 
course  development,  no  activity  provides  1ST  in  all  specific  function  items 
while  eight  schools  provide  no  inservice  in  any  of  the  items.  In  the 
evaluation  area,  one  school  provides  1ST  in  all  the  items;  12  provide  no 
inservice  in  any.  Finally,  no  school  provides  1ST  in  all  service  to  activ¬ 
ity  items;  only  two  schools  provide  it  in  half  or  more  of  the  items.  Thus, 
overall  the  extent  of  inservice  provided  varies  widely  from  activity  to 
activity,  but  only  a  few  activities  provide  inservice  in  a  majority  of  the 
specific  function  items. 


TABLE  9.  AVERAGE  PERCENTAGES  OF  CISO  RESPONDENTS  WHO  REPORTED 
THAT  THEY  RECEIVED  INSERVICE  IN  MAJOR  FUNCTION  AREAS 


Officers 


Enl isted 


Civilian 


Total 


Note:  These  percents  indicate  the  percent  of  respondents  who  checked  a 

"yes"  response  of  the  total  who  answered  the  item,  averaged  over  all 
the  items  in  each  category. 


TRAINING  EXECUTIVES.  The  percentages  of  training  executives  who  reported 
"yes"  to  whether  they  received  1ST  are  shown  in  table  10.  An  average  of  10 
percent  of  training  executives  overall  reported  1ST  in  specific  functions. 
On  the  average,  commanding  officers  reported  inservice  for  7  percent  of  the 
functions;  executive  officers,  5  percent  of  the  functions;  and  directors  of 
training,  20  percent. 
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TABLE  10.  AVERAGE  PERCENTAGES  OF  TRAINING  EXECUTIVES  WHO  REPORTED 
THAT  THEY  RECEIVED  INSERVICE  IN  MAJOR  FUNCTION  AREAS 


COs 

(n=?9) 

XOs 

(n3T2) 

DOTs 

(K=2T) 

Total 

Quality  Assurance 

5 

2 

20 

9 

Personnel  Assessment 

3 

5 

27 

12 

Management  of  Throughput 

4 

3 

15 

8 

Personal  Counseling 

22 

20 

24 

22 

Overal 1 

7 

5 

20 

10 

Note: 


These  percents  indicate  the  percent  of  respondents  who  checked  a 
"yes"  response,  averaged  over  all  the  items  in  each  category. 

I 


Personal  counseling,  as  a  major  function,  receives  the  most  1ST  overall. 
However,  for  DOTs,  personnel  assessment  receives  slightly  more  1ST  than  per¬ 
sonal  counseling.  As  a  group,  DOTs  receive  more  inservice  than  either  COs 
or  XOs.  In  fact,  with  the  exception  of  personal  counseling,  COs  and  XOs 
receive  very  little  1ST.  Generally,  the  amount  of  training  provided  training 
executives  is  less  than  the  amount  provided  CISO  personnel. 

Most  activities  provide  no  1ST  at  all  for  training  executives.  Specific¬ 
ally,  about  two-thirds  do  not  provide  inservice  in  quality  assurance,  personnel 
assessment,  or  management  of  student  throughput.  About  half  do  not  provide 
inservice  in  personal  counseling. 

The  extent  of  1ST  for  training  executives  in  the  12  areas  of  knowledge 
was  equally  scanty.  On  the  average,  training  executives  report  inservice  in 
10  percent  of  the  areas  of  knowledge.  However,  1ST  in  the  areas  of  knowledge 
does  appear  to  be  more  widespread  than  inservice  in  functions.  More  than 
two-thirds  of  the  activities  provide  inservice  in  at  least  some  of  the  areas 
of  knowledge. 

COMPARISONS  OF  FREQUENCY  OF  PERFORMANCE 

Differences  in  the  frequency  of  performance  of  a  function  indicate  dif¬ 
ferent  kinds  of  jobs.  However,  differences  in  difficulty  of  performance  may 
indicate  nothing  more  than  the  different  abilities  of  individuals  in  the 
same  job.  Likewise,  the  reported  usefulness  of  1ST  would  perhaps  indicate 
only  whether  or  not  an  individual  felt  the  need  for  improvement  in  a  particular 
area.  Thus,  frequency  of  performance  was  used  to  determine  if  the  three 
major  types  of  personnel  in  CIS  offices  (civilians,  officers,  and  enlisted) 
had  different  kinds  of  jobs  and  if  the  three  types  of  training  executives 
(COs,  XOs,  and  DOTs)  had  different  kinds  of  jobs. 
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Figures  1  through  4  present  bar  graphs  that  show  the  differences  in 
frequency  of  performance  among  the  three  types  of  CISO  personnel  for  their 
four  major  function  groups.  In  these  figures,  civilians  are  broken  down 
into  education  specialists  (n=34)  and  training  specialists  (n=14).  Figures 
5  through  8  present  bar  graphs  showing  the  differences  among  the  three  types 
of  training  executives.  In  all  the  figures,  the  vertical  axis  is  the  fre¬ 
quency.  The  values  given  correspond  to  the  numbers  of  the  response  options 
for  the  frequency  question  on  the  questionnaire  (appendix  A).  The  response 
options  range  from  daily  to  never  with  an  arbitrary  midpoint  set  at  every  3 
months. 

Care  is  required  in  interpreting  the  frequency  means.  No  absolute 
assignments  should  be  made  (i.e.,  that  someone  does  something  every  3  months 
and  others  do  it  only  every  6  months).  The  only  safe  inference  is  that  some 
perform  a  function  more  often  than  others.  This  ordinal  relationship  is 
adequate  for  the  purpose  of  this  study,  which  was  to  determine  an  order  of 
priority.  Thus,  from  figure  1,  for  example,  the  only  conclusion  should  be 
that  education  specialists  perform  preliminary  development  functions  more 
than  enlisted  personnel  and  officers,  not  that  they  perform  the  functions 
with  any  particular  frequency. 

CISO  FREQUENCIES.  Figure  1  shows  the  relative  frequencies  of  performance  of 
the  preliminary  course  development  functions  for  civilians,  education  special¬ 
ists,  training  specialists,  enlisted  personnel,  and  officers.  Education 
specialists  report  a  significantly  higher  frequency  of  performance  than  that 
reported  by  enlisted  personnel  or  officers. 

8y  visual  examination  of  figure  1,  it  is  apparent  that  education  special¬ 
ists  also  report  more  frequent  performance  of  preliminary  development  func¬ 
tions  than  do  training  specialists.  The  difference,  correcting  for  simultane¬ 
ous  inference  using  the  least  significant  difference  technique  (Steel  &  Torrie, 
1960),  was  not  statistically  significant.  However,  given  an  interest  in 
ordinal  relationships  only  and  considering  the  small  number  of  training 
specialists  (n=14),  it  is  probably  safe  to  conclude  that  education  special¬ 
ists  do  perform  this  function  more  frequently  than  do  training  specialists. 

From  figure  2,  enlisted  personnel  perform  the  secondary  development 
functions  more  frequently  than  do  officers  or  training  specialists.  Also, 
education  specialists  report  a  frequency  very  close  to  that  of  enlisted  per¬ 
sonnel.  The  evaluation  functions  (figure  3)  are  most  frequently  performed 
by  education  specialists.  However,  none  of  the  differences  between  personnel 
qroups  are  significant.  Figure  4,  concerning  service  to  activity  functions, 
shows  that  education  specialists  report  the  highest  frequency. 

In  summary,  the  education  specialists  appear  to  be  involved  principally 
in  preliminary  development,  evaluation  and  service  to  activity  functions; 
enlisted  personnel  have  more  to  do  with  secondary  development;  officers  seem 
to  be  involved  primarily  with  evaluation  functions;  and  training  specialists, 
while  too  few  in  number  to  make  strong  statements  about,  seem  generally  simi¬ 
lar  to  enlisted  personnel.  Since  education  specialists,  enlisted  personnel 
and  officers  perform  differently,  it  appears  useful  to  consider  them  sepa¬ 
rately.  Training  specialists  are  excluded  mainly  because  there  are  so  few 
of  them.  The  remainder  of  the  analysis  of  CISO  results  presented  focus  on 
the  three  groups  mentioned  above. 
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Note:  Enlisted  frequency  significantly  higher  than  Training  Specialists  or  Officers. 

Frequency  of  Performance  of  Secondary  Development  Functions 
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te:  Education  Specialist  frequency  significantly  higher  than  Officers  or  Enlisted  Personnel. 

4.  Frequency  of  Performance  of  Service  to  Activity  Functions 
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TRAINING  EXECUTIVE  FREQUENCIES.  From  figure  5,  DOTs  show  a  significantly 
higher  frequency  of  performance  of  quality  assurance  functions  than  do  XOs. 
Commanding  officers  fall  in  between  DOTs  and  XOs.  A  similar 
relationship  is  observed  for  personnel  assessment  functions  (figure  6). 

Directors  of  training  also  report  the  most  frequent  involvement  with 
managing  student  throughput  functions  (figure  7).  Commanding  and  executive 
officers  are  quite  close  in  reported  frequency  of  managing  throughput.  Per¬ 
sonal  counseling  (figure  8)  is  the  most  frequently  performed  function  for 
all  three  groups  of  personnel,  but  the  differences  among  the  groups  are  not 
significant. 

DEVELOPING  PRIORITIES  FOR  INSERVICE  TRAINING 

Overall  priorities  for  inservice  training  for  CISO  personnel  and  train¬ 
ing  executives  were  established  based  on  the  major  functions  of  these  two 
qroups  of  personnel.  The  procedures  used  and  the  results  obtained  are  given 
below. 

Scales  that  measure  various  attributes  (e.g.,  frequency,  difficulty  and 
importance  of  performance,  and  usefulness  of  inservice)  have  been  established 
for  the  four  major  CISO  function  areas:  preliminary  course  development, 
secondary  course  development,  evaluation,  and  service  to  activity.  Based  on 
differences  in  frequency  of  performance  of  the  four  major  function  areas, 
education  specialists,  officers,  and  enlisted  personnel  have  essentially 
different  jobs.  Thus,  the  order  of  importance  of  inservice  training  was 
determined  separately  for  each  of  these  three  CISO  groups.  Likewise,  similar 
scales  have  been  established  for  the  four  major  function  areas  of  training 
executives:  quality  assurance,  personnel  assessment,  management  of  through¬ 
put,  and  personal  counseling.  Also,  based  on  differences  in  frequency  of 
performance,  COs,  XOs,  and  DOTs  have  different  jobs;  consequently,  they  are 
treated  separately  in  subsequent  analyses. 

RANKING  ALGORITHM.  To  establish  priorities  for  inservice  traininq,  a  ranking 
algorithm  was  developed.  Four  kinds  of  information  were  used  to  rank  order 
(or  prioritize)  the  major  function  areas:  (1)  frequency  of  performance,  (2) 
difficulty  of  performance,  (3)  importance  of  performance,  and  (4)  usefulness 
of  inservice.  Difficulty  of  performance  upon  arrival  was  so  highly  correlated 
with  difficulty  of  performance  now  (the  mean  correlation  for  the  four  CISO 
functions  was  0.74;  for  the  four  training  executive  functions  it  was  0.84) 
that  it  added  no  significant  new  information. 

The  selection  of  a  ranking  algorithm  is,  in  some  sense,  arbitrary  since 
there  is  no  set  rule  to  apply.  Thus,  while  a  description  of  an  algorithm  is 
presented,  as  well  as  a  rationale  for  its  use,  it  must  be  acknowledged  that 
there  is  no  right  algorithm,  only  ones  that  have  more  or  less  intuitive  appeal. 

The  first  step  in  designing  an  algorithm  was  to  determine  if  some  of 
the  scales  used  were  more  important  than  others.  Intuitively,  it  would  seem 
that  frequency  of  performance  is  less  crucial  to  priority  of  inservice  than 
the  other  three  scales.  Thus,  the  ranking  algorithm  should  give  less  weight 
to  frequency  of  performance  than  to  difficulty,  importance,  or  usefulness. 
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Figure  6.  Frequency  of  Performance  of  Personnel  Assessment  Functions  for  Training  Ex 


DAILY  1.0 


DAILY  1.0 


Technical  Report  144 


The  next  consideration  was  whether  or  not  there  should  be  different  weiqhts 
applied  to  difficulty,  importance,  or  usefulness.  A  function  that  is  diffi¬ 
cult  to  perform,  but  not  too  important,  may  need  inservice  as  much  as  a  func¬ 
tion  that  is  more  important  but  less  difficult.  Thus,  importance  and  diffi¬ 
culty  can  be  traded  off  with  one  another.  Similarly,  an  important  and  diffi¬ 
cult  function  with  low  usefulness  of  training  needs  inservice  as  much  as  a 
function  with  high  importance,  low  difficulty  and  high  usefulness.  Therefore, 
all  three  were  given  equal  weight.  In  order  to  give  importance,  difficulty 
and  usefulness  more  weight  than  frequency,  they  were  multiplied  together, 
and  frequency  was  added  to  the  product.  The  algorithm  used  to  develop  overall 
ranks  for  inservice  priority  is: 

Priority  =  (Importance  X  Difficulty  X  Usefulness)  +  Frequency. 

The  numbers  used  in  this  formula  were  the  rank  orders  of  the  means  of  the 
importance,  difficulty,  usefulness,  and  frequency  scales  for  each  of  the 
four  major  function  areas  for  each  of  the  three  groups  of  CISC  and  training 
executive  personnel.  Table  11  shows  these  rank  orders  for  CISO  personnel 
and  table  12  shows  them  for  training  executives. 

These  rank  orders  were  obtained  by  ranking  the  means  for  each  of  the 
function  scales  within  each  group  of  personnel.  Thus,  from  table  11,  educa¬ 
tion  specialists  report  preliminary  development  as  the  most  frequently  per¬ 
formed  function,  evaluation  as  the  second  most  frequently  performed  function, 
and  secondary  development  and  service  to  activity  are  tied  for  third  place 
(the  3.5  comes  from  averaging  3  and  4).  The  most  difficult  function  for 
education  specialists  is  service  to  activity;  second  is  preliminary  develop¬ 
ment;  and  tied  for  third  are  secondary  development  and  evaluation.  Similarly, 
the  rank  orders  for  officers  and  enlisted  personnel  can  be  determined  by 
lookinq  under  the  column  labelled  with  a  single  letter  ("F"  indicates  fre¬ 
quency,  "D"  indicates  difficulty,  "I"  indicates  importance,  and  "U"  indi¬ 
cates  usefulness)  under  the  label  referring  to  a  specific  personnel  group. 

Rank  ordering  is  done  across  the  four  function  areas  for  a  specific  question. 
Rank  orders  for  training  executives,  in  table  12,  show  that  the  most  fre¬ 
quently  performed  function  is  persona1  counseling.  For  commanding  and  execu¬ 
tive  officers,  the  most  difficult  function  is  personnel  assessment;  for  DOTs, 
it  is  quality  assurance.  Other  relative  ranks  can  be  determined  by  examining 
table  12. 

OVERALL  PRIORITY.  To  determine  overall  priority,  the  ordering  algorithm  is 
applied  across  the  rankings  for  a  specific  function  area.  These  overall 
rankings  for  each  function  area  are  then  compared  to  determine  the  priority 
for  1ST  for  each  of  the  three  groups  of  CISO  and  training  executive  personnel. 
To  calculate  the  overall  ranks  for  functions  for  a  group  of  personnel,  the 
ranks  of  each  function  for  the  four  questions  used  (from  table  11  or  12)  are 
entered  into  the  ordering  algorithm.  For  example,  the  score  for  preliminary 
development  for  education  specialists  would  be  (2  X  4  X  4)  +  1  =  33.  The 
"2"  is  the  rank  order  difficulty  of  preliminary  development,  the  "4's"  are 
the  rank  orders  of  importance  and  usefulness  and  the  "1"  is  the  rank  order 
of  frequency.  Similarly,  the  score  for  secondary  development  for  education 
specialists  is  35;  their  score  for  evaluation  is  5.5  and  their  score  for 
service  to  activity  is  7.5.  Since  higher  priority  is  indicated  by  a  lower 


38 


FUNCTIONS  BY  PERSONNEL  TYPE  FOR  CISO  PERSONNEL 


Technical  Report  144 


1 

ro 

OJ 

rH 

00 

s 

S- 

ro 

CVJ 

c—H 

CD 

O 

•r- 

4- 

9H  K? 

4- 

C\J 

ro 

•O' 

f — 1 

o 

8 

OsJ 

f-H 

ro 

lo 

LO 

CD 

• 

• 

C 

ZD 

ro 

rH 

i — 1 

c 

LaJ 

o 

Q_ 

CO 

LO 

LO 

>- 

S- 

• 

• 

1 — 

CD 

►— f 

ro 

f-H 

f— H 

Q_ 

LU 

T3 

CO 

LO 

Z 

CD 

• 

• 

2: 

+-> 

Q 

ro 

*— H 

rH 

o 

CO 

uo 

cl: 

LO 

LO 

LU 

• 

• 

Q_ 

U_ 

ro 

rH 

r— H 

CO 

■*-> 

CO 

>r— 

=3 

ro 

f-H 

OJ 

rO 

*—* 

ro 

r-H 

CNJ 

</0 

LT> 

LO 

c 

• 

• 

o 

o 

ro 

ro 

ro 

rH 

•r— 

4-J 

ro 

lo 

LO 

U 

• 

• 

3 

u_ 

rH 

ro 

OvJ 

ro  S 

T3 

LU 

+-> 

c 

<D 

E 

C 

CL 

CD 

>i 

O 

E 

4-> 

CL 

•r— 

CD 

O 

> 

> 

r— 

•i— 

<D 

CD 

-4-> 

O 

> 

U 

CD 

< 

>> 

O 

c 

o 

ro 

>> 

o 

4-» 

C 

•r- 

•r“ 

ro 

4-» 

CD 

E 

"O 

ro 

CJ 

•r* 

C 

3 

•r— 

r— 

o 

r— 

> 

CD 

CJ 

rO 

L- 

S- 

CD 

> 

CD 

o. 

CO 

LU 

CO 

CJ 

c 
CD 
3 
C 7 
CD 
S- 


>> 

CD 

> 


U 

CD 

CL 

CO  • 

CD  CD 
<J 


*  > 
CD  S- 
4->  CD 
rO  %/) 
CJ  c 
•r*  *r- 
TD 

C  4- 
•r-  O 


C/l  CO 

CD  co 
CL  CD 
>> 


CD  CD 
C  co 
C  3 
O 

CO  X2 

S-  C 
CD  CO 
CL 

CD 
CD  CJ 

-C  c 

H->  fO 

E 

i- 

CD  O 
“O  4- 
C  t- 
3  CD 
CL 


XJ  o 
c 

ro  (D 

CJ 
*-h  c 
ro 


•*  4-> 

O  L_ 
O 
*  CL 


U-  E 


■C 

CD  T3 
r—  C 
f—  03 


CD 

JO  * 
•O  >, 


CO  3 
C  U 
E  ■*- 
3  4- 


O  *r- 
C-)  TD 


CD 

4-» 

O 


39 


Technical  Report  144 


Technical  Report  144 


number,  the  lowest  number  overall  indicates  the  highest  overall  priority. 

Thus,  for  education  specialists,  the  overall  priority  for  inservice  traininq, 
in  descending  order,  is  evaluation,  service  to  activity,  preliminary  develop¬ 
ment,  and  secondary  development.  The  identical  prodedure  is  followed  for 
CIS  officer  and  enlisted  personnel  and  for  the  three  types  of  training  execu¬ 
tives.  The  resulting  overall  priorities  for  inservice  traininq  are  shown  in 
tables  13  and  14. 

Table  13  shows  the  overall  rankings  of  the  four  function  areas  for  each 
group  of  CISO  personnel.  Evaluation  receives  the  highest  priority  for  educa¬ 
tion  specialists;  service  to  activity  is  second;  and  preliminary  and  second¬ 
ary  development  are,  respectively,  third  and  fourth.  Evaluation  is  also  the 
highest  priority  for  officers,  followed  in  order  by  secondary  development, 
service  to  activity,  and  preliminary  development.  For  enlisted  personnel, 
secondary  development  and  evaluation  are  exactly  tied  for  highest  priority. 
However,  in  the  TAEG's  opinion,  somewhat  more  emphasis  should  be  given  to 
secondary  development  for  enlisted  personnel.  Although  evaluation  is  an 
important  topic  for  enlisted  personnel,  evaluation  of  courses  should  be  done 
primarily  by  individuals  who  are  somewhat  removed  from  the  actual  design  of 
the  course.  Preliminary  development  and  service  to  activity  are  third  and 
fourth. 

Table  14  shows  the  overall  rankings  of  the  four  function  areas  for  each 
group  of  training  executives.  Quality  assurance  receives  the  highest  priority 
for  COs  and  DOTs.  Personal  counseling  has  the  highest  priority  for  XOs. 

For  COs,  personnel  assessment,  personal  counseling  and  management  of  through¬ 
put  are  second,  third,  and  fourth  in  priority,  respectively.  For  XOs,  quality 
assurance  is  second  in  priority  and  personnel  assessment  and  management  of 
throughput  are  tied  for  third.  For  DOTs,  personal  counseling,  personnel 
assessment,  and  management  of  throughput  are  second,  third,  and  fourth, 
respectively. 
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TABLE  13.  OVERALL  PRIORITY  OF  FUNCTIONS  FOR 

INSERVICE  TRAINING  FOR  CISO  PERSONNEL 


Function! 

Type  of  Personnel 

Education 

Specialist 

Officer 

Enlisted 

Preliminary  Development 

3 

4 

3 

Secondary  Development 

4 

2 

1 

Evaluation 

1 

1 

2 

Service  to  Activity 

2 

3 

4 

*See  table  2  for  descriptions  of  the  major  function  areas. 


TABLE  14.  OVERALL  PRIORITY  OF  FUNCTIONS  FOR 

INSERVICE  TRAINING  FOR  TRAINING  EXECUTIVES 


Type  of  Personnel 

Function^ 

Commanding 

Officer 

Executive 

Officer 

Director  of 
Training 

Quality  Assurance 

1 

2 

1 

Personnel  Assessment 

2 

3 

3 

Management  of  Throughput 

4 

3 

4 

Personal  Counseling 

3 

1 

2 

lSee  table  3  for  descriptions  of  the  major  function  areas. 
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SECTION  V 
DISCUSSION 

This  section  presents  a  discussion  and  interpretation  of  the  results 
presented  in  the  previous  section.  The  limits  on  the  interpretation  and  use 
of  the  results  are  also  discussed. 

MAJOR  FUNCTION  GROUPS 

Major  function  groups  were  derived  empirically  for  CISO  and  training 
executive  personnel.  These  major  function  groups  are  discussed  below. 

CISO  FUNCTIONS.  The  jobs  of  CISO  personnel  comprise  four  major  function 
groups: 

•  Preliminary  development  of  curriculum 

•  Secondary  development  of  curriculum 

•  Evaluation 

•  Service  to  activity. 

These  four  major  function  groups  can  be  considered  as  the  major  tasks  of  the 
CIS  office.  While  it  is  possible  that  some  individuals,  particularly  in 
larger  CISOs,  may  specialize  in  one  or  two  of  these  areas,  it  is  also  possible 
that  personnel  may  be  involved  in  all  four.  However,  these  tasks  represent 
relatively  discrete  components  that  constitute  the  majority  of  the  mission 
of  the  CIS  office.  Thus,  any  1ST  program  designed  to  improve  the  performance 
of  CISO  personnel  should  address  each  of  these  four  areas. 

There  are  three  major  types  of  personnel  who  work  in  CISOs:  officers, 
enlisted  personnel  and  civilians.  Of  the  civilians,  education  specialists 
constitute  the  larqest  single  group.  There  are  significant  differences  among 
the  three  groups  in  how  frequently  they  perform  the  major  CISO  functions. 

Thus,  each  group  was  considered  separately  in  determining  the  order  of  priority 
for  1ST  in  each  of  these  four  functional  areas. 

Some  general  observations  can  be  made  concerning  the  differences  in  the 
types  of  functions  that  officers,  enlisted  personnel,  and  education  specialists 
perform.  Education  specialists  are  most  frequently  involved  in  the  performance 
of  preliminary  curriculum  development  functions.  These  functions  correspond 
roughly  to  the  analyze  and  design  phases  of  ISD.  Education  specialists 
perform  these  functions  more  often  than  they  perform  any  other  function,  and 
they  also  perform  these  functions  more  often  than  other  CISO  personnel  do. 
Education  specialists  also  report  more  frequent  performance  of  evaluation 
functions  than  officers  or  enlisted  personnel,  and  evaluation  is  the  second 
most  frequently  performed  task  for  education  specialists.  Thus,  one  could 
characterize  education  specialists,  generally,  as  performing  preliminary 
curriculum  development  and  evaluation  functions.  Training  specialists 
appear  to  function  quite  similarly  to  enlisted  personnel. 


< 


43 


Technical  Report  144 


Secondary  curriculum  ievelopment  functions,  corresponding  to  the  design 
phase  of  ISD,  are  primarily  the  province  of  enlisted  personnel,  although 
education  specialists  are  also  involved.  Enlisted  personnel  in  CISOs  are 
probably  subject  matter  experts.  There  is  an  apparent  partnership  between 
them  and  education  specialists  for  the  production  of  curriculum.  The  educa¬ 
tion  specialists  perform  the  more  general  preliminary  development  functions 
while  the  enlisted  personnel  perform  the  more  specific  secondary  development 
functions.  This  is,  of  course,  a  general  statement  and  there  will  be  local 
variations.  Also,  there  is  some  involvement  of  enlisted  personnel  in  prelim¬ 
inary  development  functions  and  considerable  involvement  of  education  special¬ 
ists  in  secondary  development. 

The  most  prevalent  functions  for  officers  are  in  evaluation.  In  fact, 
the  other  three  functions  do  not  represent  a  major  component  of  activity  for 
officers.  It  may  be  that  officers  tend  to  be  more  involved  with  the  general 
management  of  operations  than  with  the  performance  of  specific  CISO  functions. 
However,  the  evaluation  functions  can  certainly  be  considered  of  interest  to 
general  managers  as  well  as  to  professional  educators.  For  this  reason, 
inservice  training  for  officers  should  perhaps  concentrate  on  how  to  evaluate 
education  and  training  programs. 

TRAINING  EXECUTIVE  FUNCTIONS.  The  jobs  of  training  executives  can  be  divided 
into  four  major  function  groups: 

•  Quality  Assurance 

•  Personnel  Assessment 

•  Management  of  Student  Throughput 

•  Personal  Counseling. 

These  four  major  function  groups  can  be  considered  as  the  major  tasks  of 
training  executives,  just  as  the  four  major  CISO  functions  mentioned  earlier 
can  be  considered  the  major  tasks  of  the  CIS  office.  Inservice  for  training 
executives  should  address  each  of  these  four  areas. 

There  are  three  distinct  types  of  training  executives:  commanding  offi¬ 
cers,  executive  officers,  and  directors  of  training.  As  was  the  case  with 
CISO  personnel,  these  three  types  of  personnel  were  considered  separately. 

The  most  frequently  performed  functions  for  all  three  types  of  training  execu¬ 
tives  are  in  the  area  of  personal  counseling.  Although  this  counseling  activity 
is  no  doubt  important,  one  must  ask  whether  or  not  such  frequent  involvement 
represents  optimal  use  of  these  officers'  time. 

RANKING  FUNCTIONS  OVERALL 

An  ordering  algorithm  was  used  to  rank  major  functions  overall,  in  terms 
of  priority  for  inservice  training,  for  each  of  the  three  major  groups  of 
personnel  for  both  CISOs  and  training  executives.  The  specific  algorithm 
used  is  described  in  section  IV.  Again,  it  must  be  emphasized  that  the  choice 
of  an  algorithm  is  largely  arbitrary.  This  particular  algorithm  was  chosen 
to  give  greater  relative  weights  to  importance  and  difficulty  of  performance 
and  usefulness  of  inservice  training.  Frequency  of  performance  was  also 
used  but  was  weighted  less  than  the  other  three  factors. 
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As  described  in  section  IV,  importance,  difficulty,  and  usefulness  were 
given  a  greater  weight  by  multiplying  them  together,  while  frequency  was 
added  to  the  product.  The  resultant  number  indicates  overall  rank.  An 
alternative  method  would  have  been  to  give  each  factor  a  weight,  perhaps  by 
multiplying  some  by  two  or  three,  and  adding  up  those  numbers.  However,  the 
weights  chosen  would  also  have  been  arbitrarily  selected.  The  method  chosen 
was  considered  as  minimizing  arbitrary  choice  and  remaining  as  close  to  the 
original  ranks  of  the  four  major  function  areas  as  possible. 

The  application  of  this  ordering  algorithm  to  CISO  responses  led  to  the 
overall  rankings  in  table  13.  For  education  specialists,  the  1ST  needs,  in 
descending  order  of  priority,  are  evaluation,  service  to  activity,  preliminary 
development,  and  secondary  development.  For  officers,  the  training  needs 
are  evaluation,  secondary  development,  service  to  activity,  and  preliminary 
development.  For  enlisted  personnel,  the  training  needs  are  secondary  develop¬ 
ment,  evaluation,  preliminary  development  and  service  to  activity. 

Although  each  group  of  CISO  personnel  has  a  different  order  of  priority 
for  1ST,  evaluation  stands  out  as  important  to  all  groups.  One  could  interpret 
this  to  indicate  that  evaluation  is  the  weak  point  in  the  abilities  of  person¬ 
nel  assigned  to  training  activities.  Thought  should  be  given  to  a  program 
desiqned  to  correct  this  weakness.  The  area  of  evaluation  is  highly  technical 
and  is  not  a  skill  that  is  easily  or  automatically  acquired. 

The  application  of  this  ordering  algorithm  to  training  executive  responses 
led  to  the  overall  rankings  in  table  14.  For  commanding  officers,  the  inservice 
training  needs,  in  descending  order  of  priority,  are  quality  assurance,  personnel 
assessment,  personal  counseling  and  management  of  student  throughput.  For 
executive  officers,  the  training  needs  are  personal  counseling,  quality  assur¬ 
ance,  and  personnel  assessment  and  managing  throughput  tied  for  third.  For 
directors  of  training,  the  training  needs  are  quality  assurance,  personal 
counseling,  personnel  assessment,  and  management  of  throughput. 

LIMITATIONS  ON  INTERPRETING  RESULTS 

The  type  of  questionnaire  data  used  in  this  study  should  not  be  used  to 
make  absolute  judgments  about  frequency,  importance,  or  difficulty  of  perfor¬ 
mance  or  the  usefulness  of  inservice  training.  The  interest  in  this  study 
was  primarily  in  establishing  ordinal  relationships.  The  data  are  more  than 
adequate  for  this  purpose.  To  make  other  interpretations  would  be  to  misuse 
the  results  of  this  study. 

Two  other  limiting  factors  in  this  study  should  also  be  mentioned.  First, 
the  entire  study  is  based  on  what  personnel  are  currently  doing  in  their 
jobs.  It  was  beyond  the  scope  of  the  tasking  to  determine  if  the  current 
functions  are,  in  fact,  the  most  desirable  ones  to  be  performed.  Second, 
the  list  of  functions  used  on  the  questionnaire  was  only  representative,  not 
exhaustive.  A  questionnaire  that  included  all  possible  functions  that  could 
be  performed  by  personnel  in  CISO  billets,  or  by  training  executives,  would 
be  prohibitively  lengthy. 


Technical  Report  144 


Even  with  these  limitations,  the  ordinal  relationships  are  meaningful. 
Given  the  functions  that  personnel  currently  perform  in  CISO  billets,  and 
their  reported  difficulty  and  importance,  the  most  efficient  inservice  train¬ 
ing  program  for  each  of  the  three  groups  of  personnel  would  be  one  that 
emphasized  training  in  areas  indicated  as  having  the  highest  priority  and 
gave  relatively  less  emphasis  to,  but  did  not  ignore,  areas  indicated  as 
havinq  lower  priorities.  Given  the  functions  that  training  executives  cur¬ 
rently  perform,  most  emphasis  should  be  given  to  inservice  in  quality  assur¬ 
ance  functions.  Next  should  come  personal  counseling,  followed  by  personnel 
assessment  and  managing  throughput. 

Training  executives  would  also  benefit  from  some  inservice  training  in 
certain  areas  of  knowledge.  Commanding  officers  express  a  need  for  inservice 
in  the  formal  NAVEDTRACOM  planning  and  reporting  systems  (e.g.,  NITRAS,  PPBS) 
and  in  how  training  is  conducted  (e.q.,  ISD,  organization  of  the  NAVEDTRACOM) 
Executive  officers  report  a  need  for  inservice  in  reporting  systems,  manpower 
planning  and  Civil  Service  rules  and  regulations.  Directors  of  training 
report  their  primary  needs  in  specific  training  related  areas,  including 
ISD,  curriculum  approval  and  training  equipment  support. 
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SECTION  VI 

CONCLUSIONS  AND  RECOMMENDATIONS 


CONCLUSIONS 

There  are  several  general  conclusions  to  this  study.  First,  inservice 
training  is  not  given  uniformly  across  the  NAVEDTRACOM.  The  extent  of 
inservice  varies  greatly  across  training  activities.  Even  where  there  is 
1ST,  it  often  addresses  only  some  of  the  functions  covered  in  this  study. 
Second,  the  instructions  that  deal  with  inservice  training  in  the  NAVEDTRACOM 
are  quite  vague  as  to  the  content.  Although  some  flexibility  should  be  left 
to  local  activities  in  providing  1ST,  there  should  perhaps  be  greater  speci¬ 
ficity  in  the  required  coverage  of  1ST.  Third,  there  are  three  distinct 
types  of  CISO  personnel  and  training  executives  who,  in  most  cases,  have 
distinct  1ST  priorities  that  reflect  the  differing  natures  of  their  jobs. 

All  CISO  personnel  should  have  inservice  training  in  evaluation  func¬ 
tions.  Officers  need  inservice  training  primarily  in  evaluation,  but  should 
also  have  an  introduction  to  the  other  functions.  Education  specialists 
need  1ST  in  service  to  activity  functions.  Enlisted  personnel  need  1ST  most 
in  secondary  development  functions. 

All  training  executives  should  have  1ST  in  quality  assurance  functions 
as  a  first  priority.  Personal  counseling  and  personnel  assessment  also  seem 
to  be  likely  candidates  for  inservice.  Managing  student  throughput  can  at 
least  be  given  reduced  emphasis  and  it  may  require  no  1ST  at  all.  All  train¬ 
ing  executives  would  benefit  from  some  introduction  to  the  various  reporting 
systems  and  to  manpower  planning.  Some  introduction  to  principles  of  training 
and  to  the  training  command  itself  is  also  indicated. 

RECOMMENDATIONS 

The  recommendations  presented  below  are  based  on  the  assumption  that 
the  personnel  involved  will  continue  to  perform  functions  reported  as  typical 
for  their  classification.  There  may  be  local  exceptions.  If,  for  example, 
there  is  no  civilian  education  specialist  at  an  activity,  then  it  is  likely 
that  an  officer  or  senior  enlisted  person  will  assume  those  functions  that 
would  normally  be  delegated  to  the  education  specialist.  In  this  case,  the 
officer  or  enlisted  person  should  receive  1ST  as  if  he  or  she  were  an  educa¬ 
tion  specialist.  Breaking  CISO  personnel  into  three  groups  should  work  quite 
well  in  most  cases.  However,  in  all  cases,  an  individual  should  receive  1ST 
based  on  what  he  or  she  does,  not  on  what  he  or  she  is.  Since  the  types  of 
training  executives  are  based  on  specific  positions  occupied,  these  caveats 
do  not  apply. 

The  following  recommendations  are  also  offered. 

1.  Instructions  requiring  1ST  at  training  activities  should  specify  a 
minimally  acceptable  level/content  of  training.  Also,  a  single  bi llet/position 
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at  each  training  activity  should  be  responsible  for  ensuring  that  inservice 
training  is  done.  The  most  reasonable  position  to  assign  this  responsibility 
to  is  the  CIS  officer  or  senior  education  specialist. 

2.  A  general  introductory  level  orientation  to  all  the  functions  that 
CISOs  perform  should  be  given  to  all  CISO  personnel  as  they  arrive.  If 
changes  occur  in  the  functions  or  requirements,  through  changes  in  policy, 
instructions  or  workload,  then  additional  introductory  level  inservice 
should  be  given. 

3.  For  all  CISO  personnel,  there  should  be  fairly  extensive  1ST  in 
evaluation  functions.  This  is  likely  to  be  a  continuing  requirement.  All 
CISO  personnel  should  know  how  to  evaluate  a  training  program,  including  the 
strengths  and  weaknesses  inherent  in  such  evaluations.  They  should  also 
know  how  external  evaluation  is  conducted  for  the  NAVEDTRACOM  and  how  that 
information  can  be  used  to  improve  training. 

4.  Education  specialists  should  also  be  given  1ST  in  the  service  to 
activity  functions  (see  table  2).  These  functions  deal  primarily  with 
assessing  needs  for  and  delivering  inservice  training  at  the  local  level; 
and  coordinating  with  external  activities.  Enlisted  personnel  should 
receive  further  inservice  training  in  secondary  development  functions.  CIS 
officers  probably  do  not  need  inservice  training  beyond  the  general 
introduction  and  the  evaluation  training. 

5.  Prospective  commanding  and  executive  officers  of  training 
activities  should,  as  a  matter  of  policy,  be  sent  to  the  prospective 
Commanding  Officer  Shore  Station  Management  Course  offered  by  the  Naval 
Civilian  Personnel  Command,  if  it  is  not  already  policy  to  do  so. 

6.  All  training  executives  should  be  given  inservice  in  quality 
assurance  functions,  reporting  systems,  and  manpower  planning  as  a  minimum. 

If  additional  1ST  is  to  be  provided,  the  priorities  discussed  previously 
should  be  used  as  a  guide. 
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APPENDIX  A 
QUESTIONNAIRES 


This  appendix  contains  a  copy  of  the  cover  letter,  distribution 
procedures,  and  questionnaires  given  to  all  CISO  personnel  and  traini 
executives. 
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DEPARTMENT  OF  THE  NAVY 
TRAINING  ANALYSIS  AND  EVALUATION  GROUP 

ORLANDO.  FLORIDA  32813 


TAEGtERH 

W1371 


From:  Director,  Training  Analysis  and  Evaluation  Group 
To:  Commanding  Officer 

Subj:  In-Service  Training  Needs  Assessment 
Ref:  (a)  CNET  ltr  Code  022  of  25  Sep  81  (NOTAL) 

Enel:  (1)  Questionnaire  Package 

1.  The  Training  Analysis  and  Evaluation  Group  (TAEG)  has  been  tasked 
(reference  (a))  to  conduct  a  study  to  identify  and  prioritize  the  in- 
service  training  needs  of  instructor  and  staff  personnel  assigned  to 
NAVEDTRACOM  training  activities.  The  information  developed  will  be  used 
by  CNET  in  planning  for  the  acquisition  and  delivery  of  training  needed  to 
upgrade  local  manpower  quality. 

2.  The  CNET  tasking  requires  that  the  TAEG  obtain  specific  information 
from  various  training  activity  personnel.  This  information  is  being 
collected  via  questionnaires.  Your  support  of  the  TAEG  effort  and 
encouragement  of  your  staff  for  a  maximum  questionnaire  return  rate  is 
requested. 

3.  Enclosure  (1)  contains  survey  forms  (questionnaires)  designed  for 
obtaining  required  information.  It  is  requested  that  these  forms  be 
distributed  to  individuals  occupying  bi llets/positions  identified  on 
each  questionnaire  cover  sheet.  Detailed  instructions  for  distributing 
questionnaires  are  contained  within  the  questionnaire  package. 

4.  To  preserve  anonymity  of  respondents  and  to  promote  candid  responses 
and  comments,  it  is  requested  that  individuals  complete  the  forms  indepen¬ 
dently  and  return  them  directly  to  the  TAEG  in  the  envelopes  provided. 
Completed  forms  should  be  returned  witvn  10  working  days  after  receipt. 

/s/ 

A.  F.  SMODE 


Copy  to:  (w/o  end) 
CNTECHTRA  (N63P) 
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QUESTIONNAIRE  DISTRIBUTION  PROCEDURES 

Four  sets  of  questionnaires  are  contained  within  the  attached  package. 

Each  set  is  intended  for  a  different  group  of  personnel.  Please  distribute 
the  questionnaires  within  your  activity  in  accordance  with  the  information 
below. 

1.  One  copy  each  of  the  Training  Executive  survey  should  go  to  the  activity 
Commanding  Officer,  the  Executive  Officer,  and  the  Director  of  Training  (if 
you  have  a  DOT). 

2.  The  form  marked  Training  Managers  should  qo  to  half  of  the  training 
department  heads,  school /course  heads,  and  all  other  individuals  who  function 
as  supervisors  of  assigned  instructors. 

3.  CISO  Personnel  survey  forms  should  be  given  to  al 1  those  military  and 
civilian  personnel  assigned  to  the  CISO  who  perform  curriculum  development 
or  evaluation  work.  Personnel  assigned  to  CISO  who  function  purely  in  cler¬ 
ical  (e.g.,  general  typing)  or  administrative  (e.g.,  student  control)  efforts 
need  not  complete  questionnaires.  If  any  CISO  personnel  function  in  dual 
capacities  (e.g.,  the  DOT  or  a  department  head  is  also  the  CIS  Officer), 
please  determine  what  job  represents  the  major  part  of  that  individual's 
effort  and  give  him/her  the  corresponding  questionnaire. 

4.  The  questionnaire  marked  for  Instructors  is  intended  to  go  to  about  10 
percent  of  assigned  personnel  who  are  currently  functioning  as  instructors. 

This  includes  personnel  from  other  services  who  are  teaching  Navy  courses. 
However,  it  does  not  include  contract  instructors.  It  is  important  that  the 
instructors  and  training  managers  designated  by  your  activity  to  complete  a 
questionnaire  be  randomly  selected.  This  means  that  all  instructors  assigned 
to  the  activity  should  have  an  equal  chance  of  being  included  in  the  sample 
regardless  of  how  long  they  have  been  there,  whether  they  are  considered 

good  or  bad  and  irrespective  of  the  particular  course(s)  they  currently  teach. 
Likewise,  every  traininq  manager  should  have  an  equal  chance  of  being  selected. 

One  way  to  select  a  random  sample  of  instructors  is  to  pick  a  number 
between  0  and  9  and  give  a  questionnaire  to  each  instructor  whose  SSN  ends 
in  that  number.  This  should  give  you  a  random  sample  of  about  10  percent  of 
your  instructors.  A  random  sample  of  training  managers  could  be  selected  by 
giving  a  questionnaire  to  each  training  manager  whose  SSN  ends  in  an  odd 
number.  Any  questions  may  be  addressed  to  the  following  TAEG  personnel  at 
the  autovon  numbers  listed: 

Larry  Ford  791-4367 

Gene  Hall  791-5673 

Gene  Michel i  791-5198 

5.  If  we  have  sent  too  many  questionnnaires  for  any  group  of  people,  please 
have  them  returned  to  TAEG  with  a  notation  on  the  cover  sheet  of  each 
questionnaire  that  they  are  excess.  Just  the  word  "EXCESS"  or  "EXTRA"  will 
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do.  This  will  allow  us  to  keep  an  accurate  record  of  response  rate.  Con¬ 
versely,  if  we  did  not  send  enough  questionnaires  for  any  group  of  people, 
please  call  and  let  us  know  how  many  more  you  need. 

These  procedures  are  important  because  they  allow  us  to  determine  how 
accurate  the  information  we  get  is.  Since  decisions  about  inservice  training 
will  be  made  based  on  the  information  you  provide,  it  is  important  that  it 
be  as  accurate  as  possible. 
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INSERVICE  NEEDS  ASSESSMENT  SURVEY 
FOR 

CISO  PERSONNEL 

The  Training  Analysis  and  Evaluation  Group,  at  the  request  of  CNET,  is 
conducting  a  study  to  assess  the  inservice  training  needs  of  training 
activity  personnel.  This  survey  is  part  of  that  study. 

Those  of  you  who  participated  in  the  CISO  study,  which  the  TAEG  recently 
completed,  may  notice  some  similarity  in  this  questionnaire.  The  results  of 
that  study  were  used  to  help  design  this  study.  Your  continued  cooperation 
is  appreciated. 

This  survey  is  to  be  filled  out  by  all  CISO  personnel  at  each  activity 
selected  who  are  directly  involved  in  instructional  support  efforts. 
Individual  responses  will  not  be  disclosed  to  anyone  outside  of  TAEG.  The 
number  in  the  upper  right  corner  of  this  page  will  be  used  to  identify 
training  activities  and  to  keep  track  of  questionnaires  returned.  It  will 
not  be  used  to  tie  questionnaires  to  individual  respondents.  All  data  will 
be  summarized  in  appropriate  tables  and  charts. 

Please  return  the  completed  form  directly  to  the  TAEG  in  the  attached 
envelope.  If  you  have  any  questions  please  call  Larry  Ford,  autovon  791 - 
4367,  Gene  Hall,  autovon  791-5673,  or  Gene  Michel i,  autovon  791-5198. 
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INSERVICE  TRAINING  NEEOS  ASSESSMENT  SURVEY  FOR  CISO  PERSONNEL 

INSTRUCTIONS 

This  survey  form  is  intended  for  CISO  personnel.  The  form  is  divided 
into  2  sections.  Complete  the  form  independently  of  other  personnel.  Brief 
instructions  for  each  section  are  given  below. 

Section  I.  This  section  asks  for  information  concerning  your  educational 
and  work  background.  Please  answer  each  question  as  accurately  as  possible. 

Section  II.  This  section  deals  with  functions  that  CISO  personnel  may 
perform.  The  functions  are  listed  down  the  left  side  and  questions  for  each 
function  are  listed  across  the  top  of  each  page.  Every  person  in  a  CIS 
Office  will  not  necessarily  be  involved  in  each  function.  Further,  CISO 
personnel  may  be  involved  in  the  performance  of  a  function,  but  they  may  not 
actually  perform  that  function.  Involvement  could  instead  be  in  an  advisory 
capacity  or  could  consist  of  reviewing  work  done  by  others.  Please  answer 
the  question  for  each  function  with  which  you  are  involved,  regardless  of 
the  degree  of  your  involvement. 
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INSERVICE  NEEDS  ASSESSMENT 
FOR 

ALL  CISO  PERSONNEL 

SECTION  I.  Respondent  Data 

a.  Series  (civilian)  or  NEC/NOBC  -  _ 

b.  Grade  (civilian)  or  rate/rank  (e.g.,  03,  E7,  GS-12)  -  _ 

c.  If  enlisted,  give  rating  (e.g.,  EM,  YN)  -  _ 

d.  Number  of  years/months  in  current  position  -  _  /  _ 

e.  What  is  the  highest  educational  level  you  have  attained?  (circle  one) 

(1)  did  not  graduate  from  high  school 

(2)  high  school  diploma  or  G.E.D. 

(3)  some  college  or  technical  training  beyond  high  school,  but  not 
bachelor's  degree 

(4)  graduated  from  college  (B.A.,  B.S.,  or  other  bachelor's  degree) 

(5)  some  graduate  school  but  no  graduate  degree 

(6)  Master's  degree 

(7)  Ed.S.,  Ed.D.,  Ph.D.,  or  other  post  master's  or  professional  degree 

f.  To  what  extent  have  the  requirements  of  your  job  changed  since  you 
started  in  your  current  position? 

(1)  (2)  (3)  (4)  (5) 

Not  at  To  a  To  a  To  a  To  a 

All  Slight  Moderate  Great  Very  Great 

Extent  Extent  Extent  Extent 

g.  To  what  extent  have  new  practices  or  techniques  in  instructional 
delivery  been  implemented  since  you  started  in  your  current  position? 


(1) 

(2) 

(3) 

(4) 

(5) 

Not  at 

To  a 

To  a 

To  a 

To  a 

All 

SI ight 

Moderate 

Great 

Very  Great 

Extent 

Extent 

Extent 

Extent 

h.  To  what  extent  have  new  practices  or  techniques  in  the  evaluation  of 
instruction  been  implemented  since  you  started  in  your  current  position? 

(1)  (2)  (3)  (4)  (5) 

Not  at  To  a  To  a  To  a  To  a 

All  Slight  Moderate  Great  Very  Great 

Extent  Extent  Extent  Extent 
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INSERVICE  NEEDS  ASSESSMENT  SURVEY 
FOR 

TRAINING  EXECUTIVES 


The  Training  Analysis  and  Evaluation  Group  (TAEG),  at  the  request  of 
CNET,  is  conducting  a  study  to  assess  the  inservice  needs  of  training 
activity  personnel.  This  survey  is  part  of  that  study. 

This  survey  is  to  be  filled  out  by  all  training  executives  at  each 
activity  selected.  Training  executive  is  the  term  used  to  refer  to 
Commanding  Officers,  Executive  Officers  and  Directors  of  Training. 
Individual  responses  will  not  be  disclosed  to  anyone  outside  of  the  TAEG. 
The  number  in  the  upper  right  corner  of  this  page  will  be  used  to  identify 
training  activities  and  to  keep  track  of  questionnaires  returned.  It  will 
not  be  used  to  tie  questionnaires  to  individual  respondents.  All  data  will 
be  summarized  in  appropriate  tables  and  charts. 

Please  return  the  completed  form  directly  to  the  TAEG  in  the  attached 
envelope.  If  you  have  any  questions,  please  call  Larry  Ford,  autovon  791 - 
4367,  Gene  Hall,  autovon  791-5673,  or  Gene  Michel i,  autovon  791-5198. 
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INSERVICE  TRAINING  NEEDS  ASSESSMENT  SURVEY 

FOR 

TRAINING  EXECUTIVES 

INSTRUCTIONS.  This  survey  form  is  intended  for  training  executives  (i.e., 
commanding  officers,  executive  officers,  and  directors  of  training).  The 
form  is  divided  into  3  sections.  Please  complete  the  form  independently  of 
other  personnel.  Brief  instructions  for  each  section  are  given  below. 

Section  !_•  This  sections  asks  for  information  concerning  your  educa¬ 
tional  and  work  background.  Please  answer  each  question  as  accurately  as 
possible. 

Section  II.  This  section  deals  with  functions  that  training  executives 
may  perform.  The  functions  are  listed  down  the  left  side  and  questions  for 
each  function  are  listed  across  the  top  of  each  page.  Every  training  execu¬ 
tive  will  not  necessarily  be  involved  with  each  function.  Please  read  each 
function  and  answer  the  questions  across  the  top  for  that  function  as  accur¬ 
ately  as  you  can. 

Section  III.  This  section  is  concerned  with  areas  of  knowledge  that 
training  executives  might  find  useful  in  the  performance  of  their  duties. 
These  areas  of  knowledge  do  not  necessarily  reflect  any  specific  duties  or 
functions  of  training  executives.  However,  familiarity  with  some  of  these 
areas  may  be  helpful  to  the  training  executive  in  carrying  out  other  duties 
or  responsibilities.  Please  read  each  area  carefully  and  answer  the 
questions  across  the  top  as  accurately  as  possible. 
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INSERVICE  NEEDS  ASSESSMENT  FOR 
TRAINING  EXECUTIVES 


SECTION  I.  Respondent  Data 

a.  Rank  (e.g.,  04,  06)  -  _ 

b.  Current  position  (please  circle  one) 

(1)  Commanding  Officer 

(2)  Executive  Officer 

(3)  Director  of  Training 

c.  Number  of  years/months  in  current  position  -  _  /  _ 

d.  Number  of  previous  tours  of  duty  as  a  training  executive  _ 

e.  Number  of  previous  tours  of  duty  at  a  training  activity  in  any 

capacity  (except  as  a  student)  _ 


f.  What  is  the  highest  educational  level  you  have  attained?  (circle  one) 

(1)  did  not  graduate  from  high  school 

(2)  high  school  diploma  or  G.E.D. 

(3)  some  college  or  technical  training  beyond  high  school,  but  not 
bachelor* s  degree 

(4)  graduated  from  college  (8. A.,  B.S.,  or  other  bachelor's  degree) 

(5)  some  graduate  school  but  no  graduate  degree 

(6)  Master's  degree 

(7)  Ed.S.,  Ed.O.,  Ph.D.,  or  other  post  master's  or  professional  degree 

g.  To  what  extent  have  the  requirements  of  your  job  as  a  training 
executive  changed  since  you  started  in  your  current  position? 


(1) 

(2) 

(3) 

(4) 

(5) 

Not  at 

To  a 

To  a 

To  a 

To  a 

All 

SI  ight 

Moderate 

Great 

Very 

Extent 

Extent 

Extent 

Great 

Extent 

h.  To  what  extent  have  new  practices  or  techniques  in  the  management  of  a 
training  activity  been  implemented  since  you  started  in  your  current 
position? 


(1) 

(2) 

(3) 

(4) 

(5) 

Not  at 

To  a 

To  a 

To  a 

To  a 

All 

Slight 

Moderate 

Great 

Verv 
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The  following  questions  are  for  commanding  and  executive  officers  only. 

i.  Have  you  ever  served  as  a  commanding  or  executive  officer  at  any  other 
shore  activity? 

(1)  (2) 

No  Yes 

j.  Have  you  attended  the  PCO  Shore  Station  Management  Course  offered  by 
the  Personnel  Management  Insitute,  NCPC? 

(1)  (2) 

No  '  Yes 

k.  If  so,  would  you  recommend  it  to  all  prospective  commanding  or 
executive  officers  of  training  activities? 

(1)  (2) 

No  Yes 


1.  Have  you  attended  any  other  PCO/PXO  courses? 


TRAINING  EXECUTIVE  FUNCTIONS 


TRAINING  EXECUTIVE  FUNCTIONS 


students  on 
personal  matters 


TRAINING  EXECUTIVE  FUNCTIONS 
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Determine  corrective  (1)  (2)  (3)  (4)  (S)  (6)  (1)  (2) 

action  If  a  criterion 
falls  be  ton  desirable 
level 


TRAINING  EXECUTIVE  FUNCTIONS 
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Students  on 
personal  matters 


AREAS  OF  KNOWLEDGE 
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D.  Did  you  receive  any  formal  Inservice  E.  tf  not,  how  useful  would  inservice 

training  In  this  area  when  you  first  training  in  this  area  have  been  when  you 

arrived  at  this  activity?  first  arrived  at  this  activity? 
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This  appendix  presents  the  results  of  factor  analyses  of  data  concerning 
the  frequency  of  performance  of  CISO  and  training  executive  functions. 

Principal  components  factor  analysis  with  varimax  rotation  was  performed 
on  the  frequency  of  performance  items  for  CISO  and  training  executive  person¬ 
nel.  The  resulting  factor  pattern  matrix  (or  loadings)  are  shown  in  table 
B-l  for  CISO  personnel  and  in  table  B-6  for  training  executives.  An  item 
that  loads  highly  on  a  factor  can  be  assigned  to  that  factor.  All  the  items 
loading  highly  on  a  factor  indicate  what  that  particular  factor  measures. 

Thus,  for  example,  the  10  items  under  preliminary  development  in  table  B-l 
constitute  factor  I,  since  they  all  load  more  highly  on  that  factor  than  on 
any  other  factor.  Similarly,  secondary  development  items  load  on  factor  IV, 
evaluation  items  load  on  factor  III  and  service  to  activity  items  load  on 
factor  II.  Similarly,  there  are  four  factors  for  training  executives. 

Each  set  of  items  that  loads  on  the  same  factor  can  be  considered  as 
measuring  the  same  dimension  or  factor.  In  this  case,  there  are  four  dimen¬ 
sions,  or  four  major  function  areas,  that  are  represented  by  four  sets  of 
items  each  for  CISOs  and  training  executives.  These  sets  of  items  were  com¬ 
bined  by  averaging  each  respondent's  score  over  all  the  items  in  the  set  to 
get  a  score  that  represents  the  factor. 

For  CISO  personnel,  each  set  of  items  that  loaded  on  a  factor  was  then 
factor  analyzed  separately  using  an  oblique  rotation.  An  oblique  rotation 
allows  for  correlation  between  factors  extracted.  An  orthogonal  rotation 
would  necessarily  result  in  only  one  factor  extracted  from  each  set  of  items. 
The  results  of  the  oblique  rotations  for  CISOs  are  shown  in  tables  B-2  through 
B-5. 


Table  B-2  shows  the  results  of  the  oblique  factor  analysis  of  the  pre¬ 
liminary  development  items.  There  are  two  subfactors,  as  can  be  determined 
by  examining  the  loadings.  The  first  five  items  load  on  factor  II  and  the 
next  five  items  load  on  factor  I.  Tables  B-3  through  B-5  can  be  examined  in 
a  similar  way.  In  each  case,  there  are  two  subfactors,  indicating  that  each 
of  these  major  function  areas  has  two  subordinate  components. 

For  purposes  of  analysis,  the  major  function  areas  were  used.  However, 
for  purposes  of  designing  curricula  for  inservice  training,  it  could  be  use¬ 
ful  to  consider  these  various  subfactors  as  parts  of  a  course  of  training 
designed  to  address  the  major  function  area. 
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Items 

Factors 

PRELIMINARY  DEVELOPMENT 

I 

II 

III 

IV 

A.  Analyze  job/new  requirements 

.73 

.13 

.40 

.05 

B.  Select  tasks  to  be  trained 

.77 

.26 

.27 

.25 

C.  Select  job  performance  measures 

.65 

.19 

.26 

.25 

D.  Analyze  existing  courses/modules 

.76 

.16 

.32 

.05 

E.  Select  instructional  setting 

.63 

.39 

.17 

.15 

F.  Develop  learning  objectives 

6.  Develop  test  items  to  measure 

.80 

.17 

.00 

.25 

objectives 

.66 

.23 

.11 

.41 

H.  Describe  entry  level  behavior 

.72 

.35 

.07 

.24 

I.  Establish  sequence  of  objectives 

.83 

.22 

.13 

.31 

J.  Specify  learning  events/activities 

SECONDARY  DEVELOPMENT 

.80 

.30 

.10 

.30 

K.  Specify  instruction  management 

plan  and  media 

.40 

.30 

.37 

.63 

L.  Select  existing  materials 

.40 

.22 

.15 

.67 

M.  Develop  instruction 

.42 

.13 

.15 

.75 

N.  Validate  instruction 

.26 

.06 

.39 

.63 

0.  Assist  with  student  testing 

.08 

.30 

.26 

.63 

EVALUATION 

P.  Study  attrition  and  setbacks 

S.  Design  instruments  for  internal 

.30 

.34 

.54 

.45 

evaluation 

.22 

.38 

.61 

.47 

T.  Collect  data  for  internal  evaluation 

.15 

.19 

.84 

.25 

U.  Analyze  data  from  internal  evaluation 

V.  Make  recommendations  based  on 

.19 

.22 

• 

00 

.19 

internal  evaluation 

.13 

.54 

.65 

.06 

Q.  Assist  with  preparation  of  items 

for  external  evaluation 

R.  Assist  with  interpretation  of 

.32 

.34 

.59 

.44 

results  of  external  evaluation 

SERVICE  TO  ACTIVITY 

.28 

.36 

.62 

.35 

W.  Design  interdepartmental  inservice 

programs 

X.  Conduct  interdepartmental  inservice 

.24 

.79 

.29 

.26 

programs 

Y.  Assist  with  intradepartmental 

.23 

.78 

.20 

.27 

inservice  programs 

.19 
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.22 
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TABLE  B-l.  VARIMAX  ROTATED  FACTOR  LOADINGS  OF  FREQUENCY 

OF  PERFORMANCE  ITEMS  FOR  CISO  PERSONNEL  (continued) 


Items 

Factors 

I 

II 

III 

IV 

z. 

Coordinate  with  external  activities 

and  agencies  concerned  with 

training  quality  assurance 

.36 

.61 

.17 

.08 

AA. 

Coordinate  accreditation  require- 

ments  and  review 

.41 

.67 

.24 

.26 

BB. 

Advise  commanding  officer  on  CISO 

civilian  affairs 

.34 

.72 

.23 

.08 
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TABLE  B-2.  OBLIQUE  ROTATION  OF  FACTOR  PATTERN  MATRIX  FOR  FREQUENCY 
OF  PERFORMANCE  OF  CISO  PRELIMINARY  DEVELOPMENT  FUNCTIONS 


Items 


Factors 


PRELIMINARY  DEVELOPMENT 

I 

II 

Analyze  job/new  requirements 

-.09 

.84 

Select  tasks  to  be  trained 

.18 

.72 

Select  job  performance  measures 

-.08 

.87 

Analyze  existing  courses/modules 

.24 

.50 

Select  instructional  setting 

.06 

.62 

Develop  learning  objectives 

.92 

-.10 

Develop  test  items  to  measure  objectives 

.55 

.14 

Describe  entry  level  behavior 

.69 

.07 

Establish  sequence  of  objectives 

.91 

.01 

Specify  learning  events/activities 

.90 

-.01 

NOTE:  Correlation  between  factors  I  and  II  is  .57. 


TABLE  B-3.  OBLIQUE  ROTATION  OF  FACTOR  PATTERN  MATRIX  FOR  FREQUENCY 
OF  PERFORMANCE  OF  CISO  SECONDARY  DEVELOPMENT  FUNCTIONS 


Items 

Factors 

SECONDARY  DEVELOPMENT 

I 

II 

1.  Specify  instruction  management 

plan  and  media 

.73 

.11 

Select  existing  materials 

.74 

.01 

Develop  instruction 

.87 

.20 

Validate  instruction 

.69 

.13 

2.  Assist  with  student  testing 

.03 

.98 

NOTE:  Correlation  between  factors  is  .20. 


Items 


Factors 


EVALUATION 

I 

II 

1.  Study  attrition  and  setbacks 

.57 

.06 

Design  instruments  for  internal  evaluation 

.49 

.35 

Collect  data  for  internal  evaluation 

.85 

.02 

Analyze  data  from  internal  evaluation 

Make  recommendations  based  on 

.93 

-.03 

internal  evaluation 

2.  Assist  with  preparation  of  items  for 

.93 

-.10 

external  evaluation 

Assist  with  interpretation  of  results 

.01 

.92 

of  external  evaluation 

-.01 

.89 

NOTE:  Correlation  between  factors  is  .55. 


TABLE  B-5.  OBLIQUE  ROTATION  OF  FACTOR  PATTERN  MATRIX  FOR  FREQUENCY 
OF  PERFORMANCE  OF  CISO  SERVICE  TO  ACTIVITY  FUNCTIONS 


Items 

Factors 

SERVICE  TO  ACTIVITY 

I 

II 

1.  Design  interdepartmental  inservice  programs 
Conduct  interdepartmental  inservice  programs 
Assist  with  intradepartmental  inservice 
programs 

.83 

.90 

.88 

.06 

.03 

.01 

2.  Coordinate  with  external  activities  and 
agencies  concerned  with  training 
quality  assurance 

.01 

.83 

Coordinate  accreditation  requirements 
and  review 

-.09 

.81 

Advise  commanding  officer  on  CISO 
civilian  affairs 

.09 

.46 

NOTE:  Correlation  between  factors  is  .34. 
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